Difference among CRT, microTVM and uTVM

Questions
#1

Hi,

I’ve been using TVM to deploy pretrained models to Bare Metal and Linux devices.

When I was going through a document and the source code, I get confused about CRT, microTVM and uTVM.

My understanding is that:

  • CRT (C Runtime) is a TVM runtime for Bare metal and RTOS devices. It doesn’t require the devices to have a dynamic library loading feature.
  • microTVM runtime is also a TVM runtime for Linux (, macOS and Windows in the future?) devices. It requires the devices to have dynamic library loading feature.
  • uTVM (or microTVM) is a way to deploy pretrained model to Bare metal and RTOS devices. It include such as TVM compiler, runtime (such as CRT which I mentioned above), AutoTVM, etc.

Is my understanding correct?

Thanks.

#2

hi @mshr-h, thanks for your post! let me correct a couple of points:

  • CRT: you’re correct, but CRT can also be used with linux if so desired. CRT is less featured than the C++ runtime, so if you have linux, it’s mostly better to use C++ runtime. apps/bundle_deploy demonstrates both.
  • microTVM runtime: on bare metal, microTVM uses the CRT. You can run microTVM with a host computer attached and send commands to drive inference on the edge device over e.g. UART. When run in this mode, there are some common components used to facilitate communication. The directory you linked to contains source code to enable TVM on the host computer to talk to the edge device.
  • uTVM is an older name for microTVM which we’re deprecating (consolidating to microTVM for clarity)
  • The rest of what you said about uTVM/microTVM is correct.

Andrew

1 Like
#3

In addition there is the Ahead of Time (AoT) work which enables you to deploy networks to bare metal and embedded deployments without the need for a host to be connected to the embedded device.

Ramana

#4

Hi @areusch and @ramana-arm

Thank you for your response! I’ll also check AoT later.

Masa

#5

Hi, thank you for your response!

Could you drop me a pointer where I can try with the Ahead of Time feature? I want to cross-compile a model to an Arm 32 bits binary (to deploy on a bare-metal device). Thank you in advance!

#6

@wuruoyu please see tests/python/relay/aot/test_crt_aot.py. AoT is still being actively developed so things may be a bit unstable across revisions, but should pass at main.

#7

Thanks for your quick response.

Yes, I have tried with tests/python/relay/aot/test_crt_aot.py and it works well. For instance, I can build a binary for mobilenet.

But I am not sure about how to do cross-compilation (to ARM 32 bits). Could you give some hints? Thanks in advance!

#8

@wuruoyu ah my apologies…see also tests/micro/zephyr/test_zephyr_aot.py.