What is ‘target’ in TVM?

[areusch]

@sho hi there, sorry for the confusion. hopefully I can clear some of this up.

@sho

Target should be used to mean something like host, iPhone, raspberry pi, but why is it ‘llvm’?

You’re absolutely right that there is some ambiguity we need to address here. In general, “target” in TVM is intended to describe the deployment environment. In practice, the context behind target often means it takes on slightly more nuanced meanings:

1. When used in tvm.relay.build, it does describe the deployment environment, but it’s described through the lens of “which codegen should we use to build for this environment?” Therefore it’s currently possible to describe environments using multiple different target strings.
2. When used with AutoTVM, it describes the configuration of the codegen which was used to produce the timed operator implementation. Most times, this matches the deployment environment, but in particular with microTVM we are doing some cleanups here around e.g. executor and runtime which really have no bearing on AutoTVM. you can see some of that discussion here.

@sho

If llvm is a kind of compilers (sorry if my understanding is wrong), can users even specify their own compilers to run micro TVM?

You’re right that LLVM is a suite of compiler tools, and Clang is the name of the C/C++ compiler built using that. When you specify an llvm Target to TVM, it means to translate the model implementation directly into LLVM’s Internal Representation for a particular target, bypassing C. This allows TVM to specify the most optimal code for many CPU platforms. You could also specify c target (which is another way to describe the same deployment environment, as discussed before), and TVM will instead generate C code meant to be consumed by a downstream compiler.

As you noted, some vendors have released specialized compilers. If you want to use those to compile everything including the emitted model code, you do need to specify c target to TVM. However, it’s important to note that there are quite a few pieces involved with deploying microTVM:

1. The compiled model.
2. The runtime components.
3. Your program.

The latter pieces are most often compiled with whatever compiler is typically used for your platform. However, piece #1 can often be built directly by TVM using the llvm target because the compiled model code should mainly depend on the CPU architecture in use. Since many boards use popular architectures which LLVM can target, such as ARM or RISC-V, you can often get away with specifying llvm here.

However, I will say for microTVM that for now, I suggest sticking with the c target if you a) need better visibility into what’s running on the CPU e.g. using a debugger b) want to use AOT due to a limitation of our codegen right now (we have yet to test the new embedded C interface with llvm target)

At least it’s the best starting point, for now. As the project matures, then a more general rule will start to apply here, where you will often see the best performance if you use the llvm target.

@sho

Please give me some information on what llvm is used for in TVM. Is it for compiling runtime or maybe any other purpose?

It is used to translate an abstract representation of the model implementation (i.e in TVM’s TIR language) into executable machine code.

@sho

In TVM, what does target mean and why is it llvm?

Answered above, I hope. Please let me know if you have more questions.

@sho

As for micro TVM, can users build micro TVM runtime using their own compiler?

yes

@sho

When it comes to microcontrollers, can we use micro TVM for microcontrollers produced by any vendors such as NXP, TI or Renesas? We have to give a specific string before starting the build, I’m wondering how we can use other microcontrollers.

You absolutely can. I’d suggest starting with things supported by the Zephyr RTOS or Arduino to make your life easier. While ST is working on a port specific to their AI deployment tool, you can also target their microcontrollers using the standard microTVM flow (indeed, we often test on STM-Nucleo boards).