[BYOC] How backwards compatible does the TensorRT partition_for_tensorrt function need to be?

Questions
4

Thanks Cody. It looks like the API will be:

  trt_target = tvm.target.Target("tensorrt -use_fp16=True -implicit_batch_mode=False")
  mod = partition_for_tensorrt(mod, params=params, target=trt_target)
  exe = vm.compile(mod, target=["cuda", trt_target], params=params)

(and similarly for the other build APIs).

If the default TRT options are good then there’s no need for any additional targets at all:

  mod = partition_for_tensorrt(mod, params=params)
  exe = vm.compile(mod, target="cuda", params=params)
2 Likes