Why do we need to build TVM runtime on remote device for Auto TVM?

Can I build the TVM runtime locally and then put it inside the target device? I am going to use iMX 6 DL and it has a total of only 32 MB flash.

1 Like

Yes, you can cross-compile it.

you can compile tvm runtime on any armv7 device. e.g. on RaspberryPi and then copy whl, egg or install tvm runtime to folder , zip the folder and copy zip file to iMX 6 DL.

@apivovarov iMX 6 DL has arm cortex A9 v7 and none of the Rasberry Pi available product has arm cortex A9. If I generate the Auto TVM log file from any arm v7 then will it work on iMX 6 DL as well?

1 Like

The run time should work on a Cortex-A9 as long as you are using the same ABI on the Raspberry pi and the imx6 . Further you should be using command line options that are compatible with the Cortex-A9. -march=armv7-a , -mfpu=neon , -mfloat-abi=hard .


What OS does it have? Linux / Android ?
Do you ssh <ip> or adb shell in order to get to the device shell?

Can you run the following commands on the device and give us the output?

uname -a

cat /etc/issue

getprop sys.kernel.version                                     
getprop ro.build.version.release
getprop ro.build.version.name
getprop ro.product.cpu.abi
getprop ro.product.cpu.abilist32
getprop ro.product.cpu.abilist64
getprop ro.product.cpu.abilist

cat /proc/cpuinfo

I have a stripped down version of Linux running on iMX 6 DL.
I use ssh to connect to my board.

uname -a: Linux #### 3.0.35-#### #1 SMP PREEMPT Fri Nov 10 10:15:08 CET 2017 armv7l GNU/Linux

cat /proc/cpuinfo:

                       Processor       : ARMv7 Processor rev 10 (v7l)
                       processor       : 0
                       BogoMIPS        : 1581.05

                       processor       : 1
                       BogoMIPS        : 1581.05

                       Features        : swp half thumb fastmult vfp edsp thumbee neon vfpv3
                       CPU implementer : 0x41
                       CPU architecture: 7
                       CPU variant     : 0x2
                       CPU part        : 0xc09
                       CPU revision    : 10

                       Hardware        : Freescale i.MX 6Quad/DualLite/Solo Sabre Auto Board
                       Revision        : 61112
                       Serial          : 0000000000000000

I don’t have the /etc/issue folder so I couldn’t provide the output of cat /etc/issue.

We also need to check glibc version available on the device

Can you compile the following C program glibc_version.c
copy executable (glibc_version) to the device and run it ./glibc_version

# Use gcc cross compiler on Ubuntu to compile the program for Armv7
# sudo apt-get install gcc-arm-linux-gnueabihf
arm-linux-gnueabihf-gcc glibc_version.c -o glibc_version


#include <stdio.h>
#include <stdlib.h>
#include <gnu/libc-version.h>

int main() {
  printf("glibc version: %s\n", gnu_get_libc_version());
  return 0;

glibc_version cross compiled with arm-linux-gnueabihf-gcc is not working on my board and when I cross compiled it with arm-linux-gnueabi-gcc it gave the output as:

glibc version: 2.20

Can you double check that your device has /lib/arm-linux-gnueabi folder without hf?

As for glibc version - you can use Ubuntu 14.04 or Debian 8 (Jessie) OS to cross-compile TVM runtime for your device - both has glibc 2.19

To cross-compile TVM runtime for arm-linux-gnueabi

cd tvm
mkdir build
cd build
cp ../cmake/config.cmake .

cmake \
-DCMAKE_C_COMPILER=/usr/bin/arm-linux-gnueabi-gcc \
-DCMAKE_CXX_COMPILER=/usr/bin/arm-linux-gnueabi-g++ \
-DCMAKE_FIND_ROOT_PATH=/usr/arm-linux-gnueabi \

make -j1 VERBOSE=1 runtime

One issue here - tvm code compilation requires gcc version 5.4 or higher.
Ubuntu 14.04 or Debian 8 (Jessie) OS has gcc version 4.x

Try to compile TVM runtime on Ubuntu 16. Maybe the device will not complain about libtvm_runtime.so glibc version.

to Test TVM

copy tvm folder to the device to /opt
export LD_LIBRARY_PATH=/opt/tvm/build
export PYTHONPATH=/opt/tvm/python


import tvm
from tvm.contrib import graph_runtime

Hi @apivovarov , i tried to cross compile android runtime, as follows

  cd tvm
  mkdir arm_runtime
  cp cmake/config.cmake arm_runtime
  cd arm_runtime
  cmake .. -DCMAKE_CXX_COMPILER="/path/to/aarch64-linux-android-g++"
  make runtime

but get errors

/home/xj-zjd/work_space/self_work/tvm_code/tvm/src/runtime/thread_pool.cc: In member function 'int tvm::runtime::ParallelLauncher::WaitForJobs()':
/home/xj-zjd/work_space/self_work/tvm_code/tvm/src/runtime/thread_pool.cc:113:26: **error: 'to_string' is not a member of 'std'**
         err += "Task " + std::to_string(i) + " error: " + par_errors_[i] + '\n';

cross compiler is generater by android ndk16b,I google this error find std::to_string is a c++11 features, It means aarch64-linux-android-g++ doesnt support c++11?

I use make VERBOSE=1 runtime to check compile options ,and found “-std=c++11” is added

/home/xj-zjd/work_space/self_work/tvm_code/android-toolchain-arm64/bin/aarch64-linux-android-g++  -DDMLC_USE_FOPEN64=0 -DTVM_THREADPOOL_USE_OPENMP=0 -Dtvm_runtime_EXPORTS -I/home/xj-zjd/work_space/self_work/tvm_code/tvm/include -I/home/xj-zjd/work_space/self_work/tvm_code/tvm/3rdparty/dlpack/include -I/home/xj-zjd/work_space/self_work/tvm_code/tvm/3rdparty/dmlc-core/include -I/home/xj-zjd/work_space/self_work/tvm_code/tvm/3rdparty/rang/include -I/home/xj-zjd/work_space/self_work/tvm_code/tvm/3rdparty/compiler-rt -I/home/xj-zjd/work_space/self_work/tvm_code/tvm/3rdparty/picojson -I/home/xj-zjd/work_space/self_work/tvm_code/tvm/vta/include
  -std=c++11 -O2 -Wall -fPIC  -fPIC   -pthread -o CMakeFiles/tvm_runtime.dir/src/runtime/thread_pool.cc.o -c /home/xj-zjd/work_space/self_work/tvm_code/tvm/src/runtime/thread_pool.cc
/home/xj-zjd/work_space/self_work/tvm_code/tvm/src/runtime/thread_pool.cc: In member function 'int tvm::runtime::ParallelLauncher::WaitForJobs()':
/home/xj-zjd/work_space/self_work/tvm_code/tvm/src/runtime/thread_pool.cc:113:26: **error: 'to_string' is not a member of 'std'**
         err += "Task " + std::to_string(i) + " error: " + par_errors_[i] + '\n';

Do you know how to solve this?

Try ndk20b. Why using old ndk?

@apivovarov thanks for your reply, I will try ndk20b laterly. I copied ndk16b from colleague,and goes well with android_rpc :joy: and I want to try c++ rpc,so i need to cross compile tvm runtime in my pc.

btw, they use clang++ instead of g++ in ndk r20b now (clang version 8.0.7)

./toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android28-clang++ --version

@apivovarov thanks again, ndk20b works well.

Hi @apivovarov , Do you know how to cross compile android tvm runtime with USE_OPENCL?

I’m not sure that Android has OpenCL library in general. Android should have OpenGL ES instead.

adb shell
ls  /system/lib | grep -i gl 

Look at TFLite and OpenGL ES https://www.tensorflow.org/lite/performance/gpu_advanced

@apivovarov Thanks a lot , I have solve this in another way.