Is microTVM still supported?

tomhepworth · August 6, 2024, 10:57am

I have raised several questions about MicroTVM, including filing a bug report concerning some fundamental features. I have not received any responses. Is the project still maintained?

zrnsm · August 9, 2024, 2:00am

@tomhepworth I’m concerned about the same question.

I’m also confused about why the PyPi package is so out of date.

The stable release is 0.12.0 (https://pypi.org/project/apache-tvm).

I’m relying on a workflow right now that uses the 0.14.dev273 release to compile ONNX to C that I pieced together from the various bits of MicroTVM documentation. This works well enough for my purposes right now but I don’t have a ton of confidence in the health of the project. I’m considering trying to move to IREE but its support for microTVM-style environments is still in an early phase. See iree-bare-metal-arm (sorry limited to one link per post as a new user).

Let me take a look at the questions you raised and see if there’s anything I can add.

tomhepworth · August 9, 2024, 9:17am

Thanks for the reply, @zrnsm I will try those versions

github.com/apache/tvm

[Bug] Conflicting code is generated in lib0.c causing conflicting types for tvmgen_default_run and tvmgen_default__tvm_main__

opened 03:54PM - 24 Jul 24 UTC

closed 02:57PM - 08 Feb 25 UTC

tomhepworth

type: bug needs-triage

I am compiling a test model with the following TVMC command: ``` python -m tv…m.driver.tvmc compile test.onnx --verbose --target='c -keys=cpu' --runtime=crt --runtime-crt-system-lib=1 --executor=aot --executor-aot-interface-api=c --executor-aot-unpacked-api=1 --pass-config tir.disable_vectorize=1 --output-format=mlf --output=model_tvm.tar ``` Inside the generated `lib0.c` I get two definitions of both `tvmgen_default_run` and `tvmgen_default__tvm_main__`. This causes the following errors : - `Conflicting types for 'tvmgen_default_run'` - `Conflicting types for 'tvmgen_default___tvm_main__'` By simply removing the conflicting definition I am able to compile a working binary. In this forum post it was resolved with unpacked-api=true, this did not work for me from tvmc. https://discuss.tvm.apache.org/t/problem-with-tvmexecute-on-arduino-in-standalone-execution/15420 This bug report also experiences the issue https://github.com/apache/tvm/issues/14871 but it has gone unaddressed This bug report also mentions it but they are on tvm 0.13 https://github.com/apache/tvm/issues/17136 I also personally know colleagues who have experienced this too. ### Expected behavior Valid C code to be generated with no conflicts ### Actual behavior The following is generated: ```C .... TVM_DLL int32_t tvmgen_default_run(TVMValue *args, int *type_code, int num_args, TVMValue *out_value, int *out_type_code, void *resource_handle); ... #ifdef __cplusplus extern "C" #endif TVM_DLL int32_t tvmgen_default___tvm_main__(TVMValue *args, int *type_code, int num_args, TVMValue *out_value, int *out_type_code, void *resource_handle); static TVMBackendPackedCFunc _tvm_func_array[] = { (TVMBackendPackedCFunc)tvmgen_default_run, (TVMBackendPackedCFunc)tvmgen_default_fused_nn_contrib_dense_pack, (TVMBackendPackedCFunc)tvmgen_default_fused_reshape, (TVMBackendPackedCFunc)tvmgen_default_fused_reshape_add_nn_relu, (TVMBackendPackedCFunc)tvmgen_default___tvm_main__, }; static const TVMFuncRegistry _tvm_func_registry = { "\005\000tvmgen_default_run\000tvmgen_default_fused_nn_contrib_dense_" "pack\000tvmgen_default_fused_reshape\000tvmgen_default_fused_reshape_add_" "nn_relu\000tvmgen_default___tvm_main__\000", _tvm_func_array, }; static const TVMModule _tvm_system_lib = { &_tvm_func_registry, }; const TVMModule *TVMSystemLibEntryPoint(void) { return &_tvm_system_lib; } #include "tvm/runtime/c_runtime_api.h" #ifdef __cplusplus extern "C" { #endif __attribute__(( section(".rodata.tvm"), )) static const struct global_const_workspace { float fused_constant_let[10] __attribute__((aligned(16))); // 40 bytes, aligned offset: 0 float fused_constant_1_let[1] __attribute__((packed, aligned(16))); // 4 bytes, aligned offset: 48 } global_const_workspace = { .fused_constant_let = {-0x1.23b7a4p-2, 0x1.08fb6cp-2, 0x1.4128e2p-4, -0x1.03f552p-2, -0x1.09157ep-2, 0x1.8a4daep-4, -0x1.ffa53ep-3, -0x1.2849ep-3, -0x1.c1e5cp-3, -0x1.1a14aap-6}, .fused_constant_1_let = {-0x1.25dc64p-3}, }; // of total size 52 bytes __attribute__((section(".bss.noinit.tvm"), aligned(16))) static uint8_t global_workspace[68]; #include <tvmgen_default.h> TVM_DLL int32_t tvmgen_default___tvm_main__( void *onnx__MatMul_0, void *output0, uint8_t *global_const_workspace_0_var, uint8_t *global_workspace_1_var); int32_t tvmgen_default_run(struct tvmgen_default_inputs *inputs, struct tvmgen_default_outputs *outputs) { return tvmgen_default___tvm_main__(inputs->onnx__MatMul_0, outputs->output, ((uint8_t *)&global_const_workspace), ((uint8_t *)&global_workspace)); } #ifdef __cplusplus } #endif ; ``` ### Environment Any environment details, such as: Operating System, TVM version, etc OS: RHEL8 ``` $ python -m tvm.driver.tvmc --version [15:48:26] /home/dev/tvm/src/runtime/logging.cc:390: TVM_LOG_DEBUG enables VLOG statements in 'ir/transform.cc' up to level 1 [15:48:26] /home/dev/tvm/src/runtime/logging.cc:390: TVM_LOG_DEBUG enables VLOG statements in 'relay/ir/transform.cc' up to level 1 0.17.dev0 ``` ### Steps to reproduce ``` python -m tvm.driver.tvmc compile test.onnx --verbose --target='c -keys=cpu' --runtime=crt --runtime-crt-system-lib=1 --executor=aot --executor-aot-interface-api=c --executor-aot-unpacked-api=1 --pass-config tir.disable_vectorize=1 --output-format=mlf --output=model_tvm.tar ``` where test.onnx is simply a torch.Linear followed by a relu. ### Triage * backend:c

This is my main problem, Id be grateful for any help.

tqchen · August 9, 2024, 6:27pm

The recent development focus has been around the latest Relax development and support for emerging models. The relax development have the potential to simplify more things in bringing up new compilations, including those in micro environment. However, the current community effort on relax has been mainly on genAI workloads and there has not yet been support for bare metal environment. The origianl microTVM in legacy flow is mostly in maintainace mode.

tomhepworth · August 12, 2024, 3:12am

@tqchen So do you mean that a replacement for MicroTVM is coming eventually? In the mean time it does seem like there are problems with the basic operation of MicroTVM. Perhaps I am just using it wrong? I know of several people/companies who would like to use MicroTVM but are struggling with the issue I describe above.