Which is better, TVM or tensorrt?

Which is better, TVM or tensorrt? Now someone is pushing tensorrt. What are the advantages of TVM over tensorrt?

tvm 和tensorRT到底哪个好?现在有人在推tensorRT,TVM相比tensorRT有什么优点?

of course, tensorrt is better.
the performance of the operator implemented by tensorrt is faster than tvm.
and tensorrt supports new gpu architecture better. Nvidia will carefully tune performance on the new architecture, and they know detail information about the low level hardware.

It might be better to do benchmarking for both options, especially for your model. Too many factors can affect the end to end performance.

The main advantage of TVM though it is that offers flexibility in automatic performance optimization via AutoTVM, especially for emerging workloads. For example, as tvm gains support for 3d models all of its backend, including the nvidia one would take benefit from that. See some examples here https://tvm.apache.org/2019/04/29/opt-cuda-quantized

2 Likes

tvm is far more general. and more, you know, 安可.

TensorRT and TVM has many similar features. TensorRT do have automatic performance opptimization, and it’s very easy to use.

Though things may envolve, I think the major difference is that, TensorRT is dedicated for Nvidia platforms. If you are using Nvidia GPU/DLA, TensorRT is production ready mostly. Otherwise, try TVM or other software system.

There is no better I think, unless the better is defined specificly.

TensorRT == Nvidia GPU

TVM == All hardware platform

1 Like

I am happy to see that some 3d ops have been added to relay. But I did not find conv3d_transpose, which is often used for volumetric semantic segmentation. Hope this problem can be fixed soon.