Use cuda pointers with TVM

Questions
#1

Hi everybody, I’m trying to use TVM on a C++ project to remove a tensorRT dependency.

This project uses cuda code to process data and puts the result on cuda arrays that in the original code are feed to the tensorRT inference engine. I would like to use these arrays converted as DLTensor as input of my TVM network.

Is there a way to convert a cuda arrays to DLTensor? I’m new approaching TVM so sorry if this seems a dumb question, I’ve searched on this forum but I can’t find the answer.

Thank you

#2

It is possible and straightforward to convert a cuda array to DLTensor. Just set the DLTensor::data to the cuda array, DLTensor::ctx to be (kDLGPU, 0), and other fields too accordingly. If you have specific questions about a field, feel free to ask :slight_smile:

2 Likes
#3

So can I simply do that?

void* ptr;
cudaMalloc(&ptr, 10 * 10 * 2 * sizeof(float));
TVMArray *x{};
std::vector<int64_t> shape ={1, 10, 10, 2};
TVMArrayAlloc(&shape[0], shape.size(), kDLFloat, 32, 1, kDLGPU, 0, &x);
x->data = ptr;