[SOLVED] Using CPU OpenCL device, No OpenCL device

gasgallo · March 11, 2021, 3:00am

I’m trying to use autotvm on an android device with MALI GPU using opencl target.

The process can run, but I get a lot of warnings and performance is terrible:

[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:262: No OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:250: Using CPU OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:262: No OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:250: Using CPU OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:262: No OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:250: Using CPU OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:262: No OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:250: Using CPU OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:262: No OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:250: Using CPU OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:262: No OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:250: Using CPU OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:262: No OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:250: Using CPU OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:262: No OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:250: Using CPU OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:262: No OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:250: Using CPU OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:262: No OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:250: Using CPU OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:262: No OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:250: Using CPU OpenCL device
[02:49:13] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:262: No OpenCL device

GFLOPS are extremely low for workloads where using a simple target llvm -device=arm_cpu -mtriple=arm64-linux-android can give much better performance.

So my questions are:

why do I get those warnings?
am I actually using the MALI GPU?
why performance are not good?

P.S. the autotuning process also makes the android RPC app crash continuously, thus the whole process is extremely hard to complete and very unreliable.

gasgallo · March 11, 2021, 5:44am

I’ve just tried with a different device (Adreno GPU) and to run the android_rpc app test python script and the result is the same on both devices:

Run CPU test ...
4.1698e-05 secs/op

[05:42:14] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:250: Using CPU OpenCL device
[05:42:14] /incubator-tvm/src/runtime/opencl/opencl_device_api.cc:262: No OpenCL device
Run GPU(OpenCL Flavor) test ...
0.000118109 secs/op

So I guess it’s not device related. Any clue?

gasgallo · March 11, 2021, 11:33am

I’ve tested the C++ RPC server (tvm/apps/cpp_rpc at main · apache/tvm · GitHub) as well and result is still the same.

Apparently clGetDeviceIDs returns -1.

@FrozenGene

FrozenGene · March 11, 2021, 1:28pm

Have you built the libtvm_runtime.so with the correct arm OpenCL header and libOpenCL.so? If you want to cross build, you maybe need to turn on USE_OPENCL ON in the config.cmake and use -DOpenCL_INCLUDE_DIR=/arm/opencl_header_dir_path -D OpenCL_LIBRARY=/arm/opencl_library_path (see FindOpenCL — CMake 3.1.3 Documentation)

If you do it correctly, you still can not use OpenCL@GPU, I think you could consider writing one simple OpenCL C++ program and test it to make sure you could access GPU using OpenCL.

gasgallo · March 11, 2021, 2:01pm

@FrozenGene Thanks for the help!

USE_OPENCL is ON, and CMake can find openCL headers and library. Am I using the right ones? Good question.

For the headers, I get them from GitHub - KhronosGroup/OpenCL-Headers: Khronos OpenCL-Headers, as suggested on other tutorial and on Arm website too.

For the library, I pull it from my device. It should be fine.

I also set the INCLUDE_DIR and LIBRARY in the config.mk when building android_rpc or pass those to CMake when building cpp_rpc.

The same devices can normally run openCL workloads using other frameworks.

FrozenGene · March 11, 2021, 3:01pm

Could you write simple opencl program just use clGetDeviceIDs to see what happened? I think this situation should be debugged somehow.

gasgallo · March 12, 2021, 5:36am

Just finished testing. Everything seem to work fine in a simple openCL program.

Here’s the code I’ve used: Code to test openCL functionalities · GitHub

I’m compiling that code for Android (arm64-v8a) while including the openCL headers and linking to the same libOpenCL.so posted above. Then pushing the executable to my device and running it.

And the output logs:

[12:12:24] /project/incubator-tvm/apps/opencl_test/opencl_test.cc:151: GetPlatformIDs: SUCCESS
[12:12:24] /project/incubator-tvm/apps/opencl_test/opencl_test.cc:192: Found platform: 0xde763ed3
[12:12:24] /project/incubator-tvm/apps/opencl_test/opencl_test.cc:166: GetDeviceIDs: SUCCESS
[12:12:24] /project/incubator-tvm/apps/opencl_test/opencl_test.cc:199: Matched device0: 0x746d866668
[12:12:24] /project/incubator-tvm/apps/opencl_test/opencl_test.cc:130: GetPlatformInfo: SUCCESS
[12:12:24] /project/incubator-tvm/apps/opencl_test/opencl_test.cc:211: Matched platform: platform_id=0xde763ed3, platform_name=QUALCOMM Snapdragon(TM), device_type=gpu
[12:12:24] /project/incubator-tvm/apps/opencl_test/opencl_test.cc:220: SUCCESS

When running the same test with tvm_rpc C++ RPC server, I get the same (good) log from the device, but I still get the bad log from my host machine.

If I set USE_OPENCL=OFF for my host machine build and rebuild TVM runtime, then those bad logs disappear and the good logs are still in my external device.

So my guess is that the bad logs were coming from my host machine that (correctly) doesn’t have any openCL device, while on my target device openCL has always been working fine. Thus I think USE_OPENCL should not be set to ON when building the TVM runtime on the host machine. This is still quite confused and confusing in the documentation imho.

Does this sound right?

FrozenGene · March 12, 2021, 7:18am

Yes. Host machine’s log shouldn’t have effect on device. So, if device’s log is right, you could go ahead.

gasgallo · March 12, 2021, 7:30am

Thanks for the help!!

Panda6 · June 9, 2023, 6:53am

hi, have you already solve this problem, I meet the same problem with the same warning. And I think if we want to cross compile a model for gpu, we should not turn USE_OPENCL OFF.