What's the status of API related to MLF and target="c"?

microTVM
1

Hello!

Using quite straightforward code with Relax-based API from the main branch I was able to export a *.tar archive with *.c files looking quite similar to the ones that I was getting in MLF archive obtained trough TVMC tool in microTVM. However, there are only 5 functions corresponding to conv2d layers in the C code, while the model being converted contains 7 conv2d layers. So, I wonder, if the code could be generated not fully correctly.

Is this functionality supported and maintained or located in the “gray zone” currently?

import onnx

import tvm
from tvm import relax
from tvm.relax.frontend import detach_params
from tvm.relax.frontend.onnx import from_onnx

onnx_model_path = "models/model.onnx"
onnx_model = onnx.load(onnx_model_path)

input_name = "conv_1_input"  
input_shape = (1, 12, 40, 1)  
shape_dict = {input_name: input_shape}

mod_from_onnx = from_onnx(onnx_model, shape_dict)
mod, params = detach_params(mod_from_onnx)

target = tvm.target.Target("c")

ex = tvm.compile(mod, target)
ex.export_library("output/model_library.tar")
2

Ok, I’ve got it. The number of conv2d layers is just as expected. The ex._as_text() shows me that some conv functions should be called more that once in the main entry point, obviously with different weights.
Trying to figure out now, how to export the *.c file with main function containing correct flow.

3

Hello,

Were you able to make any progress on the main function?

Thanks!

4

Hello,
Rather not. In our case the “main” pseudocode that we are getting by the ex._as_text() looks like:

@main: call vm.builtin.check_tensor_info in: %0, i4, c[0], c[1] dst: %void
call vm.builtin.match_shape in: %0, %void, i4, i0, i1, i0, i12, i0, i40, i0, i1, c[1] dst: %void
call vm.builtin.reshape in: %0, c[2] dst: %1
call vm.builtin.alloc_storage in: %vm, c[3], i0, c[4], c[5] dst: %2
call vm.builtin.alloc_tensor in: %2, i0, c[6], c[7] dst: %3
call conv2d in: %1, c[8], %3 dst: %void
call vm.builtin.null_value in: dst: %1
...

The abbreviation vm, that is similar to the one of the VirtualMachine in tutorials, and the names of the functions around conv2d layers calls, that are mostly about memory management, makes us think those functions are expected to be runtime-specific. In general, if there were a need to use TVM from the main supported branch only, I think we would be capable to generate corresponding C-code in the “main” function by ourselves with some level of automation. However, while the microTVM installed from the v0.18.0 branch is still working, such task is not of high urgency for us.
Beside this, we are considering usage of the Tensorflow Lite for Microcontrollers. Its runtime induces some memory/computational overhead, but when built with CMSIS-NN and Helium MVE support, it allows us to get the performance measurements close to those we have with clear C TVM code.
Hope that helps in your case.

5

Thanks for the quick response!

We are in a similar situation and plan to continue using v0.18.0 for our MCU use cases for now.

Exploring the Relax flow on the side. Have made some progress with a static build of the runtime and updating the C source backend to generate code that uses the TVM global registry (vs. the module registry) to handle calls to vm builtins such as vm.builtin.alloc_tensor. Need to figure out how to auto generate the main wrapper that calls __vmtir__main and handle weights/params in the C backend.

1 Like