Dynamic Model Support

I enjoy the reading of https://arxiv.org/abs/2006.03031 which supports dyn model support in the TVM using relay vm.

However, i want to ask some quick questions:

  1. Do we have uploaded completely of Nimble code on the mainstream? Especially about the memory performance issue like this : [VM] The performance degradation of VM runtime and Dynamic Shape support compared to Graph Runtime. In paper we have described we have memory planning and heterogeneous device placement to solve this.

  2. The evaluated platforms contain arm, however it is on cloud. How about embedded platforms? Does VM could be one part of libtvm_runtime.so which could be cross compiled using cross compiler?

Thanks for the great work! I like this design solving this problem. @haichen @jroesch @zhiics

4 Likes

Same questions here. And I also want to ask: will you submit the code of kernel dispatching described in §4.5 in the paper to TVM mainstream? @haichen @jroesch @zhiics

@FrozenGene

  1. Most part of the Nimble code has been pushed to mainstream. So now you can use VM runtime on GPU without performance issue. The memory footprint though is still going to be larger than the graph runtime.
  2. I think VM is already part of libtvm_runtime.so.

@kongroo I’m still working on that part to have a more systematic mechanism.

2 Likes

Does the artificial evaluations of this paper publicly available?