Will TVM optimize operators that are newly added by developers?

Questions
#1

Hi, I’m new to TVM and ML system development. I am recently investigating how to add an operator (following this tutorial: Adding an Operator to Relay — tvm 0.13.dev0 documentation). I have a simple (perhaps noob) question in my mind: Will TVM optimize operators that are newly added by developers? I’m not familiar with TVM internal optimization techniques, so I ask for help here.

We can suppose I want to implement a cumulative product operator. Will TVM automatically optimize this operator as a primitive operator in the computation graph? Or TVM may see it as a combination of multiple off-the-shelf primitive operators and still possible to take apart it for optimization?

I would like to know more about the implementation details of TVM’s optimization. Could someone give me a hint of which part of its source code I should look at?

Thank you so much!

#2

The optimisation part when adding a new operator is the point 5. of the documentation: Hooking up Compute and Strategy with Relay.

You might not be familiar yet with the terms compute function and strategy. As a brief introduction in TVM you can describe what your operator computes with a compute function, and how to compute it with a set of scheduling primitives to apply to that function. This is a fundamental piece, also referred to as Tensor Expression (and scheduling primitives).

Now for a given target you might want to have different ways of implementing the operator (implementing as in TE + scheduling) that would yield different performances depending on the context. This is where operator strategy comes in.

I have never added a new operator in TVM but I have been navigating the documentation a lot recently, so I hope this helps.

1 Like
#3

Thank you so much for your answer. It helps me a lot.