[RFC] TVMC: Add support for µTVM

manupa-arm · February 9, 2021, 6:49am

Interesting discussions!

The only thing I see specific to that use-case is that it has an runtime “adapter” for TVM C and C++ interfaces (in bundle.c and bundle.cc) that will be used by the application and will be linked by an ad hoc (per project or application that wants to build the static runtime without a RPC server) against the model_c.o or model_cpp.o objects generated via the TVM build.relay API. Currently tvmc uses runtime.Module.export_library when generating .tar archive, but for that use-case I believe it should provide a .o instead , i.e. use runtime.Module.save (like in the example script)? If that is true I think that this document should address at least a way to specify a .so or .o (as required in that use-case). Then Makefile will call tvmc accordingly just passing the model (for instance, mobilenet0.25 ) + a flags specifying that a .o must be generated and tvmc take care of generating “.o” to be used.

Yes, you are mostly right!. I have also made a similar comment in the other RFC that we should additionally include a compiled artifact .a ( or it could be .o – don’t have a strong preference). I was hinting that we could do use export_library on the runtime.Module (which is a multi-module hierarchical tree) produced via relay.build using the correct cross compiler and target object format as .o/.a. However, I agree with @areusch that we still need the sources as Zephyr (west) and similar flows accept the direct sources to create alignment with compilation flags across all sources compiled to create final .elf/.bin/.hex.

On my comment to @manupa-arm I was just wondering if it would make sense to use a template (a different one from project template) for use-cases like the static runtime in apps/bundle_deploy , since in my understanding the code in bundle.c (and bundle.cc ) is a just a boilerplate/glue that will be used in similar use-cases. In that case a compile flag would exist specifying a static runtime (tvmc currently only generates a .so afaics) (like, --static-runtime) and the template would be copied to Model Library Format into src/ so the user could later easily use it in conjunction with other sources generated by build.relay on a Makefile. The advantage in my view, if that’s possible, would be to avoid repeating it boilerplate on each application that wants to use a static runtime.

I think its OK to bundle them in the model library as optional/helper sources but I think we should not link them in the compiled artifact (.o/.a). Yes, we would need a tvmc compile flag to specify the output type. I think we should discuss the contents of Model Library in that RFC