TVM build with OpenBLAS for arm64

Troubleshooting
1

Hi, all

I wanna know how can I use implementations of some ops, like GeMM, from OpenBLAS for arm64 target?

  1. I cross compiled the OpenBLAS for arm64 target, which located at "/opt/OpenBLAS/. "
  2. I add “set(BLAS_LIBRARY /opt/OpenBLAS/lib/libopenblas.a)” in build/config.cmake, where libopenblas.a is a symbolic link: libopenblas.a → libopenblas_armv8p-r0.3.16.dev.a.
  3. I tried to run “make runtime”, however, it failed with wrong ELF format. It seems that it treats the libopenblas.a as a lib on x86. And “make jvmpkg” will also make the runtime target, same error occurs.

So, how can I link the openblas for arm64 normally in TVM? which file should I modify?

Thanks in advance!

2

can anyone help? :smiley: :smiley:

3

I successfully compiled TVM with OpenBLAS on Ubuntu 18.04 Aarch64.

Here is and how I compiled.

Install OpenBLAS and clone TVM.

# apt install -y libopenblas-dev
# git clone --recursive https://github.com/apache/tvm
# cd tvm
# mkdir build && cd build
# cp ../cmake/config.cmake .
# vim config.cmake

Add set(USE_BLAS openblas) in config.cmake.

Compile TVM.

cmake .. -G Ninja
ninja runtime

Then, I got libtvm_runtime.so with OpenBLAS linked.

# ldd libtvm_runtime.so 
	linux-vdso.so.1 (0x0000ffffb10b6000)
	libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000ffffb0ec6000)
	libopenblas.so.0 => /usr/lib/aarch64-linux-gnu/libopenblas.so.0 (0x0000ffffb07fe000)
	libstdc++.so.6 => /usr/lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000ffffb066a000)
	libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffffb05b1000)
	libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000ffffb058d000)
	libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000ffffb0561000)
	libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffffb0406000)
	/lib/ld-linux-aarch64.so.1 (0x0000ffffb108a000)
	libgfortran.so.4 => /usr/lib/aarch64-linux-gnu/libgfortran.so.4 (0x0000ffffb0303000)
4

It looks like OpenBLAS was wrongly compiled - e.g. for x86. At the end of compilation process, there should bu summary (in my case arm32 It looks like):

make[1]: Leaving directory '/opt/openblas/exports'

 OpenBLAS build complete. (CBLAS)

  OS               ... Android             
  Architecture     ... arm               
  BINARY           ... 32bit                 
  C compiler       ... CLANG  (cmd & version : Android (7019983 based on r365631c3) clang version 9.0.9 (https://android.googlesource.com/toolchain/llvm-project a2a1e703c0edb03ba29944e529ccbf457742737b) (based on LLVM 9.0.9svn))
  Library Name     ... libopenblas_armv7p-r0.3.26.dev.a (Multi-threading; Max num-threads is 48)

And log of TVM build:

--
-- Found TVM_LLVM_VERSION=90
-- Build with LLVM 
-- Set TVM_LLVM_VERSION=90
-- Using BLAS library /opt/blas/lib/libopenblas_armv7p-r0.3.26.dev.so
-- Using BLAS header in /opt/blas/include
-- Build with contrib.random
-- Build with contrib.sort
-- Build with contrib.hybriddump
--
[ 87%] Building CXX object CMakeFiles/tvm_runtime_objs.dir/src/runtime/graph_executor/graph_executor_factory.cc.o
[ 87%] Building CXX object CMakeFiles/tvm_runtime_objs.dir/src/runtime/aot_executor/aot_executor.cc.o
[ 87%] Building CXX object CMakeFiles/tvm_runtime_objs.dir/src/runtime/aot_executor/aot_executor_factory.cc.o
[ 87%] Building CXX object CMakeFiles/tvm_runtime_objs.dir/src/runtime/contrib/cblas/cblas.cc.o
[ 87%] Building CXX object CMakeFiles/tvm_runtime_objs.dir/src/runtime/contrib/random/random.cc.o
[100%] Building CXX object CMakeFiles/tvm_runtime_objs.dir/src/runtime/contrib/sort/sort.cc.o
[100%] Built target tvm_runtime_objs