Use custom C++ code with TVM

Hi everyone! I am working on a project in which we have a custom optimized C++ code for performing depth-wise convolution. I was wondering how we could go about integrating this code with tvm, so that I can compare the performance of this code with the d2l-tvm depthwise optimized code. Essentially, I want TVM to run this code, so that my comparison includes the overhead of TVM → C++ I did go through a couple of links on the TVM website (mainly the one which says Bring your own Codegen), but I was unable to fully understand how to implement it. Any help would be appreciated!!

Yeah Bring your own codegen (BYOC) is probably what you are looking for. Have you seen this doc? How to Bring Your Own Codegen to TVM

We have BYOC examples of DNNL and ARM ethos, but in your case it would be much simpler.

cc @comaniac

It seems to me that if we are talking about just one op (i.e., depthwise conv2d) implemented in C++, then it’s much easier to directly integrate to TOPI and become an extern op just like other kernels in CuBLAS.

That would indeed be less code, but I’d say it is more invasive. BYOC also scales better if users want to use more custom ops.

I did not go through this link, thank you @masahi! However, it seems to be more complex than what we need. We will only be adding a single cpp file which performs optimized depthwise convolution. I see a comment by @comaniac which suggests an easier way. Is it possible to get a link to learn more about this method? Thank you for your replies!!

See for example how we integrate cublas:

This helps a ton! Thank you so much! It is always nice to find good, timely support!!!

1 Like

Hi again @masahi! Replying in this thread for ease.

I followed the code structure in TVM Runtime System — tvm 0.8.dev0 documentation.

Currently, my .cpp file has

In a .py file, I am trying to do

I get the following error:

I tried searching in the forum, but I was unable to find a solution that worked for me. Any ideas?

hmm it looks correct. Maybe you can try make clean and rebuild?

Also to be sure, you can put your function to the same file as cublas.cc etc

1 Like

Hmm, I did try that. Doesn’t seem to work :frowning:

This is my Makefile, maybe I missed something here:

Will try the cublas.cc trick, but I would prefer if I can keep my .cpp file separate

Oh you are trying to add your packed function outside of libtvm.so or libtvm_runtime.so? I’ve never seen this and am not sure if that is going to work, because libtvm.so needs to “see” your function cc @tqchen

Oh yes, I believe I am trying to do that. I am not sure if it is possible to re-compile the libtvm.so with my custom cpp file added, is it?

You should be able to do that by adding your cpp under src/runtime/contrib. See many examples there.

2 Likes

Thanks! Just to confirm, I need to rebuild TVM itself, right?

I apologize if I am asking too many questions, I am really new to TVM!

Yes, you should run cmake again after you add your cpp, otherwise it won’t be compiled.

1 Like