Conflict with XGBoost when Thrust is enabled

Development
#1

When USE_THRUST=ON, unknown CUDA error happened:

  File "/home/ubuntu/tvm/src/runtime/cuda/cuda_device_api.cc", line 108
CUDA: Check failed: e == cudaSuccess || e == cudaErrorCudartUnloading: unknown error

It can be reproduced with the following script

import numpy as np
import tvm
import xgboost

if __name__=='__main__':
    a_np = np.ones((1, 16))
    a_tvm = tvm.nd.array(a_np, ctx=tvm.gpu(0))

My environment: CUDA 10.0, thrust 1.9.3 XGBoost: 1.1.0 (installed with pip or from source)

I found a workaround is to import xgboost before importing TVM

#2

@hcho3 looks like another symbol conflict?

no it doesn’t look like that…

#3

@vinx13 I cannot reproduce the issue on my machine. Can you reproduce the issue in a Docker container?

#4

Unfortunately I’m not able to reproduce in a docker right now. I’ll update here if I find a way to reproduce it

#5

Met same problem when open USE_THRUST=ON without using xgboost

My environment: CUDA 10.1, thrust 1.9.5

#6

I got the same problem and filed a PR to workaround it.