# \[RFC\] Relay Containers Array/Map/String **URL:** \ **Category:** RFC\ **Created:** [August 10, 2020, 10:39am UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560 "2020-08-10T10:39:00Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![xiandi](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/xiandi/32/1918_2.png) [@xiandi](https://discuss.tvm.apache.org/u/xiandi)\ **Post date:** [August 10, 2020, 10:39am UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/1 "2020-08-10T10:39:00Z")
I am a mlsys engineer who mainly provides server support for the nlp-related model for the company. In the nlp-related business scenario, we found that tvm can provide superior tensor computing performance, so we intend to promote tvm applications within the company. ## Motivation: In addition to Tensor computing, we want to integrate more data processing operations with tvm relay to provide end-to-end solutions. Currently, TVM already provides data type support such as String/Array/Map at the Runtime level, but there is no corresponding interface at the **relay ir** , and we sincerely hope to provide corresponding support at the ir level. ## Example This is an example of a dictionary lookup that converts a text sequence into an id array and then do model inference. To Implement this function, we may need to support **String/List/Map** and some of its actions at the relay level. ``` py_vocab = {"hello": 0} def term2idx(): vocab = tvm.runtime.container.Map(py_vocab) terms = relay.var("terms", relay.ListType(relay.StringType())) idxes = relay.lookup(vocab, terms) # lookup is a op return relay.Function([terms], idxes, ret_type=relay.ListType(relay.scalar_type("int32"))) ``` ## Other Application Converting PyTorch models also requires these data structure, for example: ![image](https://discuss.tvm.apache.org/uploads/default/original/2X/2/2571f66ab231ab92b22af7081c3e925b4d549e2b.jpeg) ## Roadmap We want to do this in three stages: 1. Add String IR types and support easy operation of String 2. Add Array IR types and support easy operation of Array 3. Add MapIR types and support easy operation of Map Currently we have written a demo to increase the String type, hoping to get community support and feedback, and if possible, merge into tvm master, which can reduce branch maintenance costs here. PR LINK: [[relay][ir] add string type to relay ir by cloud-mxd · Pull Request #6242 · apache/tvm · GitHub](https://github.com/apache/incubator-tvm/pull/6242) Good day! ---
**Author:** ![xutianming](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/xutianming/32/2064_2.png) [@xutianming](https://discuss.tvm.apache.org/u/xutianming)\ **Post date:** [August 10, 2020, 12:42pm UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/2 "2020-08-10T12:42:38Z")
Good idea. I am compiling a torch model containing Dict input. ![image](https://discuss.tvm.apache.org/uploads/default/original/2X/2/2c7bc28ad29b7c7122326cacf0b0fab2d77c55c0.png)Tvm raised above error. ---
**Author:** ![xiandi](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/xiandi/32/1918_2.png) [@xiandi](https://discuss.tvm.apache.org/u/xiandi)\ **Post date:** [August 10, 2020, 1:10pm UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/3 "2020-08-10T13:10:27Z")
@tqchen @MarisaKirisame @jroesch ---
**Author:** ![junrushao](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/junrushao/32/205_2.png) [@junrushao](https://discuss.tvm.apache.org/u/junrushao)\ **Post date:** [August 10, 2020, 4:53pm UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/4 "2020-08-10T16:53:14Z")
Would you like to enhance the relay type system, adding types like string, array and map? ---
**Author:** ![xiandi](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/xiandi/32/1918_2.png) [@xiandi](https://discuss.tvm.apache.org/u/xiandi)\ **Post date:** [August 10, 2020, 5:02pm UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/5 "2020-08-10T17:02:07Z")
Yes, that will help convert torchscript model and integrate more business to tvm stack. we will need to discuss the best way to support them ---
**Author:** ![junrushao](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/junrushao/32/205_2.png) [@junrushao](https://discuss.tvm.apache.org/u/junrushao)\ **Post date:** [August 10, 2020, 5:14pm UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/6 "2020-08-10T17:14:35Z")
I can understand the motivation of adding string help for NLP-related models. Just some curious questions: 1. While embedding lookup can be implemented using a map lookup, TVM does not provide the functionality of tokenization, does it imply that tokenization is done outside but the lookup has to be done inside TVM module? Is there any specific reason that it must be done inside TVM module? 2. TVM String is using “char” as storage, and is not aimed for unicode handling. Does it imply that right now we only focus on ASCII string? 3. What is the plan for array type and map type in Relay? Do you introduce an ADT, or expand the type system to support arrays and maps, for example, support type inference with arrays and maps? Are types inside arrays and maps homogeneous? 4. How does it lower to TIR? Into an intrinsic call that invokes array indexing and map lookup? 5. Map is not inside the runtime right now, due to concerns of the binary size, but to use map in generated code, it seems that we need to move them into runtime. Is that correct? ---
**Author:** ![xiandi](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/xiandi/32/1918_2.png) [@xiandi](https://discuss.tvm.apache.org/u/xiandi)\ **Post date:** [August 10, 2020, 6:11pm UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/7 "2020-08-10T18:11:07Z")
First of all, thank you very much for your question, in my opinion: 1. Tokenizer and lookup can be implemented within the tvm through adding tokenizer op or substring operations. this is similar to torchscript. All nlp business placed inside the tvm mainly to consider the convenience of deployment and application promotion. 2. wo now fucus on utf8 string. 3. Relay has implemented List by ADT. However, we intend to expand the type because there may be performance issues with the adt list. The item types of arrays and maps are homogeneous which already meets the business needs. 4. we plan to add some relay vm instructions to support the operation on the string, array and map container. 5. yes，need map in runtime. The above is still in the discussion stage, Hope to get more support and feedback from the community, thank you! ---
**Author:** ![junrushao](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/junrushao/32/205_2.png) [@junrushao](https://discuss.tvm.apache.org/u/junrushao)\ **Post date:** [August 10, 2020, 7:11pm UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/8 "2020-08-10T19:11:47Z")
Thank you for your quick response. It certainly makes sense to me! ---
**Author:** ![jknight](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/jknight/32/979_2.png) [@jknight](https://discuss.tvm.apache.org/u/jknight)\ **Post date:** [August 10, 2020, 10:54pm UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/9 "2020-08-10T22:54:44Z")
> wo now fucus on utf8 string. I don’t follow. Are you planning on expanding support to include utf8 as well? > yes, need map in runtime. Runtime binary size implications are increasingly important to eg uTVM. CC @areusch . Could this support be optionally compiled into the runtime? Though that might get confusing when models run in some runtimes but not others. Is there anyway for a runtime to advertise what capabilities it supports and then validate models against their needed set of capabilities? ---
**Author:** ![junrushao](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/junrushao/32/205_2.png) [@junrushao](https://discuss.tvm.apache.org/u/junrushao)\ **Post date:** [August 10, 2020, 11:05pm UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/10 "2020-08-10T23:05:32Z")
Yeah I agree the point for runtime binary size, and that is the reason why I finally decide not to put `tvm::Map` into runtime before. In the case of uTVM, it implements its own pure C runtime IIRC @areusch, so won’t be affected by this proposal. ---
**Author:** ![jknight](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/jknight/32/979_2.png) [@jknight](https://discuss.tvm.apache.org/u/jknight)\ **Post date:** [August 10, 2020, 11:07pm UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/11 "2020-08-10T23:07:21Z")
But if TVM’s various runtime capabilities start to diverge, isn’t that a bad thing for TVM user experience? ---
**Author:** ![junrushao](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/junrushao/32/205_2.png) [@junrushao](https://discuss.tvm.apache.org/u/junrushao)\ **Post date:** [August 10, 2020, 11:11pm UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/12 "2020-08-10T23:11:00Z")
Good point, I agree! Another approach might be having a compilation flag to move related functionalities in/out of the runtime, for example, a CMake flag “USE\_MAP\_IN\_RUNTIME”, and if we need minimal binary size, we can turn the flag off ---
**Author:** ![xiandi](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/xiandi/32/1918_2.png) [@xiandi](https://discuss.tvm.apache.org/u/xiandi)\ **Post date:** [August 11, 2020, 2:01am UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/13 "2020-08-11T02:01:30Z")
> [@junrushao](#): > > USE\_MAP\_IN\_RUNTIME Good point, FYI ✈ ---
**Author:** ![xiandi](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/xiandi/32/1918_2.png) [@xiandi](https://discuss.tvm.apache.org/u/xiandi)\ **Post date:** [August 11, 2020, 2:24am UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/14 "2020-08-11T02:24:25Z")
> [@jknight](#): > > I don’t follow. Are you planning on expanding support to include utf8 as well? Maybe we can refer to lua ext [lutf8](https://github.com/starwing/luautf8/blob/master/lutf8lib.c) and provide some utf8 functions to relay? > [@jknight](#): > > Runtime binary size follow @junrushao proposal _Thanks you and best regards_ ! ---
**Author:** ![junrushao](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/junrushao/32/205_2.png) [@junrushao](https://discuss.tvm.apache.org/u/junrushao)\ **Post date:** [August 11, 2020, 8:28am UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/15 "2020-08-11T08:28:21Z")
UTF-8 string operations are highly non-trivial. If no other operations are needed besides look-up, a better way might be that we treat them as an opaque buffer. ---
**Author:** ![wwwxxxhhh](https://discuss.tvm.apache.org/letter_avatar_proxy/v4/letter/w/9d8465/32.png) [@wwwxxxhhh](https://discuss.tvm.apache.org/u/wwwxxxhhh)\ **Post date:** [August 11, 2020, 8:36am UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/16 "2020-08-11T08:36:03Z")
I modify the two files,but it didn’t work,it still have the problem, I think I will crazy ---
**Author:** ![xiandi](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/xiandi/32/1918_2.png) [@xiandi](https://discuss.tvm.apache.org/u/xiandi)\ **Post date:** [August 11, 2020, 8:37am UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/17 "2020-08-11T08:37:49Z")
opaque buffer is also ok to me. ---
**Author:** ![wwwxxxhhh](https://discuss.tvm.apache.org/letter_avatar_proxy/v4/letter/w/9d8465/32.png) [@wwwxxxhhh](https://discuss.tvm.apache.org/u/wwwxxxhhh)\ **Post date:** [August 11, 2020, 8:39am UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/18 "2020-08-11T08:39:29Z")
the pre-problem,I can not reply,I have only three times,do you have mail address,I need you help ,I use the method,modify two file,but it does not work,I think I will crazy,I try lots of methods,but all not succeed ---
**Author:** ![xiandi](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/xiandi/32/1918_2.png) [@xiandi](https://discuss.tvm.apache.org/u/xiandi)\ **Post date:** [August 11, 2020, 10:01am UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/19 "2020-08-11T10:01:05Z")
Maybe we can also add a UnicodeString type and then add two conversion functions: 1. encode(str: UnicodeString) -\> String 2. decode(str: String) -\> UnicodeString ---
**Author:** ![tqchen](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/tqchen/32/5_2.png) [@tqchen](https://discuss.tvm.apache.org/u/tqchen)\ **Post date:** [August 13, 2020, 12:03am UTC](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560/20 "2020-08-13T00:03:08Z")
Thanks for the proposal. There are a few things that we need to figure out besides the types themselves: - P0: mutability of the container - Right now all the containers are immutable, which also makes the relay analysis simpler. My understanding is that immutable containers are good for our usecase(as we mainly do lookup). but please confirm if that is the case - P1: the type signature of Type themselves, there are a few alternatives. Having a concrete Type for Array/Map/String is certainly one possibility. But we will need to confirm that we won’t add additional times besides these ones. and additional ones should be added through ADT [Next page](https://discuss.tvm.apache.org/t/rfc-relay-containers-array-map-string/7560.md?page=2)