As a next step, let us plan to phase out the micro flow which is mostly based on legacy. The particular component will remain available in 0.18.0 and previous releases and is not actively maintained as of now. We also hope to empower bringing back up some future examples through unity flow if there is community members who are interested in that direction.
2 Likes
I have to leave a comment to express my feelings about this.
I just saw the PR for this, and I have to say, this is a truly sad day for me. The thing that first brought me near TVM was microTVM, and the ability to target embedded devices with such a reduced runtime. I have been using it a lot during the last few years, and of course, will continue working with it.
My feeling is that, without it, TVM is not going to be used anymore in papers targeting custom accelerators, which was a very interesting niche that was previously mostly filled by TVM. Some of the features that could be used with it, like USMP or the AoT Executor where truly very amazing features, and it is sad to see I will not be able to take advantage of this using microTVM in the future. The phase out of the VTA flow takes TVM in the same direction.
I hope we can take back again the development of microTVM in the future, maybe building some bridge to/from Relax.
2 Likes
Thanks @fPecc , we would love to see relax based approach for targetting accelerators in future, hopefully the modularized flow make it even easier to do so, both in-tree and out-of-tree. There is indeed tradeoff here, however, at this point i also think bringing focus on the modern approach is critical for us to regain momentum and be sustainable for future developments. In the meantime, I would love to provide more inputs supporting discussions on how relax can help in some of these directions
@fPecc, folks,
I add here my humble experience with this topic, but only a pure personal point of view.
I used TVM in past for custom micro stuff (including experiments with custom fpga flows) and never relayed on the current micro part. I believe one can achieve his goals given the modularity of TVM, it is very easy to insert you passes or to hook in any parts of TVM internal flow without even touch upstream code (fork) or to declare a highly custom target with a wierd runtime. For micro stuff I always ended up using the native C codegen backend and passover results to my own needs, but this way it is possible to target even super-micro things like whatever 8bit u-controllers.
As another concrete example of custom HW acceleration I always enjoyed it that one even can insert verilated (from pure verilog land) blobs of block/micro-kernel and tensorize with it any ML operator without even touching upstream code, just by simple declarations of tensorizer in metaschedule for the tuning process. This is probably one of the neatest user-side feature of metaschedule (autotensorize, with it’s very intuitive template-declaration that auto-magically fits itself into operators).
As for the VTA part (again a personal opinion) I saw it as super inflexible & rigid thing, the mentioned [verilog-hw-blocks]->[autotenzorizer]->[metaschedule] approach for me yielded way much more flexibility and performance, and also the generated C code handled straight booth the HW acceleration parts on any custom soft-core cpu (having HW acceleration as pure ISA extensions).
I also think that the micro dragged in a lot (way to much) of non ml-compiler things, specific micro-runtime related headers and libraries that are quite diverse and numerous.
TVM really pioneers and keep pioneering lots of things starting early with elegant IRs (where was MLIR at that time ?) to the very neat end-to-end flow of autotune/metascheduling. I hope TVM continue keep the focus and rise the bar on these very things.
1 Like
Thanks @cbalint13 for sharing your exprerience, such kind of modular experience is indeed something we hope to enable in the new relax flow, love to continue working together and leverage relax pipeline helps to further modularize and enable more usecases like you mentioned, perhaps also they can serve good community tutorials for general flow 