[Discussion] A Technical Approach to LLMs with TVM Unity

On BYOC, yes, the unity flow actually simplifies the BYOC, see [Unity][Tutorial] TVM Unity BYOC

On the MCU support. Because a lot of focus have been on emerging workloads, the focus have been on platforms that comes with linux/windows/mac and we haven’t yet try to look at MCU that are below MB memory level. This being said, all the foundations are there, and the current build also have a compiled mode that get code into TIR.

The goal of unity is actually to enable such modularization so runtime backend can be reasonably decoupled from core compilation, e.g. as long we can lower an IRModule to TIR function calls that goes to set of C API, we should be able to run on the environment of interest and welcome contributions on these fronts

1 Like