What is ‘target’ in TVM?

[areusch]

@sho no problem, apologies if this is a bit confusing.

@sho

As you probably know, I was confused about LLVM, and how it is used. So LLVM is used to build TVM compiler. LLVM also works when you input your model into TVM compiler and get TVM’s IR. LLVM compiles the TVM’s IR, and produces executables. (I’m still not sure where gcc is used when you build TVM from source though…)

There are actually two roles LLVM could play in TVM:

1. To build TVM from source
2. As a Target backend (e.g. relay.build(..., target="llvm")), in which case TVM links against LLVM (e.g. LLVM is a static library) at compile time, and TVM then contains the code-generation pieces of LLVM and is in turn able to emit machine code to implement a model, just like LLVM does.

@sho

You don’t alway need to use LLVM. In that case, you specify ‘c’ as the target, and get your C code ready. You can use this C code, the runtime and your other programs to build with your own compiler, and run inference on MCUs.

This is correct. TVM’s compilation flow is:

1. import models into Relay
2. “lower” Relay into TIR
3. “codegen” TIR to match the target. // Target("llvm") or Target("c") mostly matters here

So the final codegen step is where TIR is translated into some type of machine code. That could be LLVM’s Intermediate Representation (which we can generate when TVM is linked against LLVM–like in kind #2 above), C source code, CUDA code, or other types.