TVM terms: relay, topi, tir, te

Questions
#1

I’ve been studying TVM for quite a few weeks, still not crystal-clear about the relationship between these items: relay, tir, topi, te. I’ll try to summarize my understanding and please correct me in my description below. Thanks in advance.

  1. Relay

Relay is the replacement for NNVM, which is a graph optimization toolset. It works on tensor expressions (te, question: is relay te same as tir te?).

Input te is either extracted from imported trained neural network or manually constructed using relay.<op>, those are so-called Relay IR, a high-level IR.

There are certain attr are defined or extracted for each relay operator. These attributes are used to later match or find best implementations in topi using so-called FTVMStrategy.

  1. tir

Tir is low-level IR before “translating” to target-specific backends. E.g. if x86 is target, then tir is first mapped to llvm IR, then x86 codegen is called to generate target binary.

  1. topi

A bridge between Relay IR and tir. A repository of operator implementations, describing what to compute and how to compute (schedule). The FTVMSTrategy mechanism allows relay operator to be mapped to highest scored implementation.

5 Likes
#2

Thanks for the summary. I noticed that relay.build runs all the way and runs codegen. Does relay explicitly convert RelayIR to tir? Is there an interface for that? (Presumably, it would be a way to see/debug what relay has done. Or maybe there is a better way to do the debugging?)

5 Likes
#3

Have you found out anything about explicit relay to tir translation. I am currently wondering, about the same question.

#4

First of all, I’m by no means expert in TVM. So just my two cents.

I believe the Relay-> Tir transform happens with so-called “lowering” process in side python/tvm/relay/backend/compile_engine.py, CompileEngine::lower(blah).

#5

good summary, see also https://tvm.apache.org/docs/dev/index.html

1 Like
#6

@cgerum @cxcxcxcx did you ever figure out how to lower from Relay to TIR?

1 Like
How to lower relay built modules?
#7

Well, as far as I found out, there is no straightforward way of lowering to tir as relay.lower. But is quite easy to add a tir level pass to relay.build.


@tvm.tir.transform.prim_func_pass(opt_level=0)
def print_tir(f, mod, ctx):
    print(f)

with tvm.transform.PassContext(
        opt_level=3, config={"tir.add_lower_pass": [(3, print_tir)]}
    ):
        lib = relay.build(mod, target=target, target_host=target_host, params=params)

For more details and a complete example have a look at: Writing a Customized Pass — tvm 0.8.dev0 documentation

2 Likes