Can't deploy the model remotely by RPC to RK3399

albert · August 3, 2018, 11:41am

Hi
I follow the step of “Deploy the Pretrained Model on ARM Mali GPU” Tutorial and find it always failed at step
“upload the library to remote device and load it”
rlib = remote.load_module(‘net.tar’)

I used the docker image by running
" sudo ./bash.sh tvmai/demo-gpu " on the host ubuntu 14.04
and the remote on the RK3399 only do git clone the TVM and “make runtime”

The main error is
TVMError: [11:09:33] /home/firefly/tvm/src/runtime/module_util.cc:34: Check failed: f != nullptr Loader of opencl(module.loadbinary_opencl) is not presented.

The error message list below:
TVMError: Except caught from RPC call: TVMCall CFunc Error:
Traceback (most recent call last):
File “/home/firefly/tvm/python/tvm/_ffi/_ctypes/function.py”, line 54, in cfun
rv = local_pyfunc(*pyargs)
File “/home/firefly/tvm/python/tvm/rpc/server.py”, line 48, in load_module
m = _load_module(path)
File “/home/firefly/tvm/python/tvm/module.py”, line 222, in load
return _LoadFromFile(path, fmt)
File “/home/firefly/tvm/python/tvm/_ffi/function.py”, line 280, in my_api_func
return flocal(*args)
File “/home/firefly/tvm/python/tvm/_ffi/_ctypes/function.py”, line 184, in call
ctypes.byref(ret_val), ctypes.byref(ret_tcode)))
File “/home/firefly/tvm/python/tvm/_ffi/base.py”, line 66, in check_call
raise TVMError(py_str(_LIB.TVMGetLastError()))
TVMError: [11:09:33] /home/firefly/tvm/src/runtime/module_util.cc:34: Check failed: f != nullptr Loader of opencl(module.loadbinary_opencl) is not presented.

Stack trace returned 9 entries:
[bt] (0) /home/firefly/tvm/build/libtvm_runtime.so(dmlc::StackTraceabi:cxx11+0x118) [0x7fa3499198]
[bt] (1) /home/firefly/tvm/build/libtvm_runtime.so(tvm::runtime::ImportModuleBlob(char const*, std::vector<tvm::runtime::Module, std::allocatortvm::runtime::Module >*)+0x182c) [0x7fa34c0484]
[bt] (2) /home/firefly/tvm/build/libtvm_runtime.so(+0x2c5d4) [0x7fa34a75d4]
[bt] (3) /home/firefly/tvm/build/libtvm_runtime.so(tvm::runtime::Module::LoadFromFile(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)+0x3d0) [0x7fa34b7958]
[bt] (4) /home/firefly/tvm/build/libtvm_runtime.so(+0x3fba4) [0x7fa34baba4]
[bt] (5) /home/firefly/tvm/build/libtvm_runtime.so(TVMFuncCall+0x74) [0x7fa349eeac]
[bt] (6) /usr/lib/aarch64-linux-gnu/libffi.so.6(ffi_call_SYSV+0x64) [0x7fa4392e60]
[bt] (7) /usr/lib/aarch64-linux-gnu/libffi.so.6(ffi_call+0xc0) [0x7fa43937b8]
[bt] (8) /usr/lib/python2.7/lib-dynload/_ctypes.aarch64-linux-gnu.so(_ctypes_callproc+0x670) [0x7fa43b2b30]

Can someone help me to solve this problem?

merrymercy · August 3, 2018, 4:25pm

You have to build the runtime on rk3399 with opencl enabled.
see newest tutorial

github.com

dmlc/tvm/blob/8056165eea6b55aa27b6bbed741f58dc3a6e639b/tutorials/nnvm/deploy_model_on_mali_gpu.py#L39-43


#   has Linux running.
# 
# Since we do compilation on local machine, the remote device is only used
# for running the generated code. We only need to build tvm runtime on
# the remote device. Make sure you have opencl driver in your board.
# You can refer to `tutorial <https://gist.github.com/mli/585aed2cec0b5178b1a510f9f236afa2>`_
# to setup OS and opencl driver for rk3399.
#
# .. code-block:: bash
#
#   git clone --recursive https://github.com/dmlc/tvm
#   cd tvm
#   cp cmake/config.cmake .
#   sed -i "s/USE_OPENCL OFF/USE_OPENCL ON/" config.cmake 
#   make runtime -j4
#
# After building runtime successfully, we need to set environment varibles
# in :code:`~/.bashrc` file. We can edit :code:`~/.bashrc`
# using :code:`vi ~/.bashrc` and add the line below (Assuming your TVM 
# directory is in :code:`~/tvm`):
#

albert · August 6, 2018, 2:51am

I follow the tutorial https://gist.github.com/mli/585aed2cec0b5178b1a510f9f236afa2
and run the Sanity Check with clpeak to check if the opencl driver works properly by using the benchmark tool clpeak.

The result shows below:

Platform: ARM Platform
Device: Mali-T860
Driver version : 1.2 (Linux ARM64)
Compute units : 4
Clock frequency : 200 MHz

Global memory bandwidth (GBPS)
  float   : 3.78
  float2  : 5.84
  float4  : 6.92
  float8  : 5.75
  float16 : 5.57

Single-precision compute (GFLOPS)
  float   : 22.84
  float2  : 44.63
  float4  : 44.26
  float8  : 41.33
  float16 : 46.02

half-precision compute (GFLOPS)
  half   : 22.85
  half2  : 45.90
  half4  : 92.57
  half8  : 91.79
  half16 : 66.33

Double-precision compute (GFLOPS)
  double   : 3.91
  double2  : 3.48
  double4  : 20.87
  double8  : 20.51
  double16 : 20.26

Integer compute (GIOPS)
  int   : 19.97
  int2  : 45.01
  int4  : 46.11
  int8  : 46.82
  int16 : 41.26

Transfer bandwidth (GBPS)
  enqueueWriteBuffer         : 4.12
  enqueueReadBuffer          : 2.47
  enqueueMapBuffer(for read) : 3263.65
    memcpy from mapped ptr   : 2.35
  enqueueUnmap(after write)  : 3234.16
    memcpy to mapped ptr     : 2.53

Kernel launch latency : 239.78 us

Does it mean that the OpenCL is ok on device RK3399 ?

I also try the newest tutorial with local_demo = False

github.com

dmlc/tvm/blob/8056165eea6b55aa27b6bbed741f58dc3a6e639b/tutorials/nnvm/deploy_model_on_mali_gpu.py#L39-43


#   has Linux running.
# 
# Since we do compilation on local machine, the remote device is only used
# for running the generated code. We only need to build tvm runtime on
# the remote device. Make sure you have opencl driver in your board.
# You can refer to `tutorial <https://gist.github.com/mli/585aed2cec0b5178b1a510f9f236afa2>`_
# to setup OS and opencl driver for rk3399.
#
# .. code-block:: bash
#
#   git clone --recursive https://github.com/dmlc/tvm
#   cd tvm
#   cp cmake/config.cmake .
#   sed -i "s/USE_OPENCL OFF/USE_OPENCL ON/" config.cmake 
#   make runtime -j4
#
# After building runtime successfully, we need to set environment varibles
# in :code:`~/.bashrc` file. We can edit :code:`~/.bashrc`
# using :code:`vi ~/.bashrc` and add the line below (Assuming your TVM 
# directory is in :code:`~/tvm`):
#

But it still failed at
L198 : rlib = remote.load_module(‘net.tar’)

TVMError: [02:29:59] /home/firefly/tvm/src/runtime/module_util.cc:34: Check failed: f != nullptr Loader of opencl(module.loadbinary_opencl) is not presented.

Can someone take a look at this problem?

merrymercy · August 6, 2018, 2:59am

Do you execute these lines (listed in the newest tutorial) on your board? You should build tvm runtime with set(USE_OPENCL ON) in config.cmake

git clone --recursive https://github.com/dmlc/tvm
cd tvm
cp cmake/config.cmake .
sed -i "s/USE_OPENCL OFF/USE_OPENCL ON/" config.cmake   ## MODIFY config.cmake
make runtime -j4

albert · August 6, 2018, 3:03am

Yes, I did that

firefly@firefly:~/tvm$ make runtime -j4
– The C compiler identification is GNU 5.4.0
– The CXX compiler identification is GNU 5.4.0
– Check for working C compiler: /usr/bin/cc
– Check for working C compiler: /usr/bin/cc – works
– Detecting C compiler ABI info
– Detecting C compiler ABI info - done
– Detecting C compile features
– Detecting C compile features - done
– Check for working CXX compiler: /usr/bin/c++
– Check for working CXX compiler: /usr/bin/c++ – works
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Detecting CXX compile features
– Detecting CXX compile features - done
– Performing Test SUPPORT_CXX11
– Performing Test SUPPORT_CXX11 - Success
– Build with RPC support…
– Build with Graph runtime support…
– Build VTA runtime with target: sim
– Found OpenCL: /usr/lib/aarch64-linux-gnu/libOpenCL.so (found version “1.2”)
– Build with OpenCL support
– Configuring done
– Generating done
– Build files have been written to: /home/firefly/tvm/build
.
.
.
[100%] Linking CXX shared library libtvm_runtime.so
make[4]: Leaving directory ‘/home/firefly/tvm/build’
[100%] Built target tvm_runtime
make[4]: Entering directory ‘/home/firefly/tvm/build’
Scanning dependencies of target runtime
make[4]: Leaving directory ‘/home/firefly/tvm/build’
[100%] Built target runtime

albert · August 6, 2018, 7:10am

It pass and success all steps now.

But I am wired about what is different between
the ’ sed -i “s/USE_OPENCL OFF/USE_OPENCL ON/” config.cmake ’
and use vim to chang the OFF/ON before .

Anyway, Thanks very much for your help!

merrymercy · August 6, 2018, 3:29am

They are the same in theory.

emelife · October 31, 2018, 2:34am

hello,I can’t use RPC way to control my board, because my host pc is at home and my firefly-rk3399 boad is in my office they are not in the same net region (or net segment).so Is there any other ways to instead of RPC.I can config mali target and build output graph /param /libraray and then export .finaly ,copy these to my rk3399 board , does the tvm(runtime) on rk3399 have some interface to load these ouput on pc without RPC way ? I am new in tvm. thanks.

eqy · October 31, 2018, 6:40pm

If you can ssh tunnel to your rk3399 you can still use RPC via a custom IP address so that RPC is not confused by the local address of the tunnel. See this argument.

zzw · April 19, 2019, 3:08am

hi, I got a simillar error, could you help me find out why…?
How to build TVM with OpenCL

zzw · April 21, 2019, 2:16pm

How did you build “OpenCL” on RK3399?

– Found OpenCL: /usr/lib/aarch64-linux-gnu/libOpenCL.so (found version “1.2”)