End-to-End flow using TIR MetaSchedule and uTVM

[Necrotos]

Hello,

I am trying to perform code-generation for an accelerator. I have defined a schedule and created the patterns to partition the Relay graph into the operations that are supported. These things individually work, but I don’t know how to put them together, and I can’t find any examples on how to achieve this.

I found the VTA example that defines Operator strategies, but that is integrated with AutoTVM and I want to use the MetaSchedule since I want to work directly on TIR and not TE. Is there a way for me to combine TIR Scheduling with MetaSchedule and microTVM and AOT compilation (target is baremetal) while using Relay, or do I need to move to Relax for this?

To illustrate a bit more, the compilation flow that I want to achieve is something like this:

Import PyTorch → Relay/Relax? → TIR → MetaSchedule to Optimize → Emit Code through microTVM AOT

I know this would work using TE and AutoTVM throught the TOPI integration, since this implemented already implemented for VTA, but using the newer tools is much more interesting to me I am curious if this is feasible while staying on Relay, or if I need to move everything to the Unity branch. More specifically, something to replace register_topi_compute is what I’m interested in.

CC: @tqchen @junrushao

[fPecc]

Hi @Necrotos ,

I am also interested in the newer tools, so I join with the same question. Been using Relay and microTVM for a while, but it would be interesting to know if there is a way of migrating this to Relax and TVM Unity.

[tqchen]

[DISCUSS] TVM Core Strategy for Operator Scheduling and Tuning

Background This doc aims to discuss how we should do operator scheduling(and optionally tuning) following the TVM Core strategy for emerging needs. Operator scheduling refers to the phase where we take high-level description of the operators and bring more performant code. This process is usually called scheduling. [Untitled] Over the course of development, we have several components. TE-compute: ability to describe high-level computations via tensor expression. TE-schedule TensorIR: IRMod…

see this post for a discussion about scheduling mechanisms(aka how to replace topi compute)

[tqchen]

One thing that is likely still missing is a codegen layer that takes the subset of relax function and generate something that works on a limited runtime, this is a place that would need some amount of community effort. but the operator scheduling blocker can be addressed likely via above approach in a even more composable way

[Necrotos]

Thanks! The missing layer for running in embedded environments is what I was afraid would be the case. Are you aware of any ongoing work in this direction? Since I will need this anyway, I would also like to contribute to any current or future efforts.

[fPecc]

Same here, I have colleagues interested in working on this topic, since we have been using uTVM a lot during the last years, but we would like to migrate everything to Relax.

[cgerum]

You can count me in as well.I am also very interested in further development of tvm for bare metal environments.