Extracting CUDA array pointer

Questions
1

Hi! I’m currently trying to build a python API for a GPU least squares solver Opt, and considering tvm as a option to glue them together. The solver’s C API directly takes in CUDA device pointer for input and output. Is there a public and reliable API in tvm to directly access CUDA pointer underneath tvm::Tensor?
Thanks!

2

CUDA pointer is held by DLTensor, not tvm::Tensor.

See here for how we pass DLTensor to cuDNN.

1 Like
3

Thank you for confirming such option exists! I have just started learning tvm, and will investigate further.

4

Hi @masahi, thanks to the link, I start to understand how does tvm interact with third party libraries. Third party functions are wrapped with PackedFunc, and on the python side, _api.extern and _intrin.call_packed are used to call PackedFunc.
I still have a few questions:

  • How does _intrin.call_packed find C++ PackedFunc?
  • How is tvm.tensor.Tensor converted to DLTensor?
  • Is it possible to write my wrapper functions outside of tvm repository, and link it at runtime?
  • Is there a documentation to related topics?

Thanks!

5
  • You don’t need to know this. C++ functions are registered to TVM via TVM_REGISTER_GLOBAL macro. Global functions are accessible from anywhere in TVM. See for example how the cudnn wrapper is registered https://github.com/dmlc/tvm/blob/master/src/contrib/cudnn/conv_forward.cc#L16.
  • For writing C++ function, you don’t need to know how tvm.tensor.Tensor relates to DLTensor. DLTensor is the runtime object that underlies tvm.tensor.Tensor (I think). It is not like “one converts to another”.
  • Your function needs to be registered via TVM_REGISTER_GLOBAL at compile time, so I don’t think that is possible.
  • No, as far as I know. You can look at examples in tvm/src/contrib.
1 Like
6

see https://docs.tvm.ai/deploy/integrate.html which might be helpful

1 Like