# Graph\_plan\_memory doesn't support nested tuples?

**URL:** <https://discuss.tvm.apache.org/t/graph-plan-memory-doesnt-support-nested-tuples/8278>\
**Category:** Questions\
**Created:** [October 25, 2020, 11:25pm UTC](https://discuss.tvm.apache.org/t/graph-plan-memory-doesnt-support-nested-tuples/8278 "2020-10-25T23:25:55Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![masahi](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/masahi/32/74_2.png) [@masahi](https://discuss.tvm.apache.org/u/masahi)\
**Post date:** [October 25, 2020, 11:25pm UTC](https://discuss.tvm.apache.org/t/graph-plan-memory-doesnt-support-nested-tuples/8278/1 "2020-10-25T23:25:55Z")

</div>

Hi, the model I’m working on has the following output:

```auto
  ...
  %1562 = (%1550, %1551, %1552, %1553, %1554, %1555, %1556, %1557, %1558, %1559, %1560, %1561);
  (%1549, %1562)
}

```

i.e., the output is a tuple where the second element is another tuple with 12 elements.

`relay.build(...)` errors on this model with the following message:

```auto
  [bt] (0) /mnt/2e797a66-fd2b-44fc-a3ba-24d7d65f2780/projects/dev/tvm/build/libtvm.so(+0x104245b) [0x7f30ec71a45b]
  File "/mnt/2e797a66-fd2b-44fc-a3ba-24d7d65f2780/projects/dev/tvm/src/relay/backend/graph_plan_memory.cc", line 86
TVMError: Check failed: tok.size() == 1U (12 vs. 1) : 

```

The error is happening when memory planner is visiting `TupleNode`:

> <https://github.com/apache/incubator-tvm/blob/main/src/relay/backend/graph_plan_memory.cc#L82-L87>

So it seems to me the memory planner is complaining about tuple of tuples? @tqchen do you have an idea what’s going on?

UPDATE: VM compilation doesn’t have this problem.

---

<div class="post-metadata">

**Author:** ![mbaret](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/mbaret/32/1906_2.png) [@mbaret](https://discuss.tvm.apache.org/u/mbaret)\
**Post date:** [October 26, 2020, 10:21am UTC](https://discuss.tvm.apache.org/t/graph-plan-memory-doesnt-support-nested-tuples/8278/2 "2020-10-26T10:21:38Z")

</div>

We (@manupa-arm) ran into this in the graph partitioner. I think in the end we were forced to introduce logic to flatten such tuples, so if a more fundamental solution can be found that would simplify our logic.

---

<div class="post-metadata">

**Author:** ![tqchen](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/tqchen/32/5_2.png) [@tqchen](https://discuss.tvm.apache.org/u/tqchen)\
**Post date:** [October 27, 2020, 12:05am UTC](https://discuss.tvm.apache.org/t/graph-plan-memory-doesnt-support-nested-tuples/8278/3 "2020-10-27T00:05:50Z")

</div>

Yes, we will need to update the code if we want to support nested tuple. Perhaps we can pass he token around also in nested tuples and unpack them.

---

<div class="post-metadata">

**Author:** ![jroesch](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/jroesch/32/161_2.png) [@jroesch](https://discuss.tvm.apache.org/u/jroesch)\
**Post date:** [October 27, 2020, 12:08am UTC](https://discuss.tvm.apache.org/t/graph-plan-memory-doesnt-support-nested-tuples/8278/4 "2020-10-27T00:08:07Z")

</div>

@masahi there is code for doing this mapping inside of the VM, if you message me on Slack we can probably figure out how to update the code, might require a bit of debugging

---

<div class="post-metadata">

**Author:** ![masahi](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/masahi/32/74_2.png) [@masahi](https://discuss.tvm.apache.org/u/masahi)\
**Post date:** [October 27, 2020, 12:29am UTC](https://discuss.tvm.apache.org/t/graph-plan-memory-doesnt-support-nested-tuples/8278/5 "2020-10-27T00:29:21Z")

</div>

Ok, thanks! I found the code Jared was probably referring to (t`ransform/memory_plan.py`, `transform/memory_alloc.py`, not sure why they are written in python). I’m going to learn about memory planning and see what I can do.

---

<div class="post-metadata">

**Author:** ![jroesch](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/jroesch/32/161_2.png) [@jroesch](https://discuss.tvm.apache.org/u/jroesch)\
**Post date:** [October 27, 2020, 12:39am UTC](https://discuss.tvm.apache.org/t/graph-plan-memory-doesnt-support-nested-tuples/8278/6 "2020-10-27T00:39:02Z")

</div>

There is a C++ helper called Linearize or FlattenTuple (can look later)

---

<div class="post-metadata">

**Author:** ![zhiics](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/zhiics/32/146_2.png) [@zhiics](https://discuss.tvm.apache.org/u/zhiics)\
**Post date:** [October 27, 2020, 12:41am UTC](https://discuss.tvm.apache.org/t/graph-plan-memory-doesnt-support-nested-tuples/8278/7 "2020-10-27T00:41:31Z")

</div>

The helpers are here:

> <https://github.com/apache/incubator-tvm/blob/98c2096f4944bdbdbbb2b7b20ccd35c6c11dfbf6/src/relay/op/memory/memory.cc#L287-L300>

---

<div class="post-metadata">

**Author:** ![masahi](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/masahi/32/74_2.png) [@masahi](https://discuss.tvm.apache.org/u/masahi)\
**Post date:** [October 27, 2020, 12:45am UTC](https://discuss.tvm.apache.org/t/graph-plan-memory-doesnt-support-nested-tuples/8278/8 "2020-10-27T00:45:52Z")

</div>

thanks, I’ll take a look

---

<div class="post-metadata">

**Author:** ![jroesch](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/jroesch/32/161_2.png) [@jroesch](https://discuss.tvm.apache.org/u/jroesch)\
**Post date:** [October 27, 2020, 3:38am UTC](https://discuss.tvm.apache.org/t/graph-plan-memory-doesnt-support-nested-tuples/8278/9 "2020-10-27T03:38:44Z")

</div>

@masahi FromTupleType is the one you probably want it takes a Type representing the layout of `expr` and returns a sequence of expressions which correspond to the linearized view of the tuple, i.e it will handle projecting nested tuples out.