[BYOC][ACL] How to offload op to ACL runtime/CL/functions

Questions
#1

Is there any way to offload op to ACL arm_compute/runtime/CL/functions?

According to the code arm_compute_lib/acl_runtime.cc, It seems that BYOC/ACL only supports NEON function.

#ifdef TVM_GRAPH_EXECUTOR_ARM_COMPUTE_LIB
#include <arm_compute/core/Types.h>
#include <arm_compute/runtime/NEON/functions/NEArithmeticAddition.h>
#include <arm_compute/runtime/NEON/functions/NEConcatenateLayer.h>
#include <arm_compute/runtime/NEON/functions/NEConvolutionLayer.h>
#include <arm_compute/runtime/NEON/functions/NEDepthwiseConvolutionLayer.h>
#include <arm_compute/runtime/NEON/functions/NEElementwiseOperations.h>
#include <arm_compute/runtime/NEON/functions/NEFullyConnectedLayer.h>
#include <arm_compute/runtime/NEON/functions/NEPoolingLayer.h>
#include <arm_compute/runtime/NEON/functions/NEReshapeLayer.h>

#include "acl_allocator.h"
#include "acl_utils.h"
#endif
#2

It is possible but requires building TVM and linking it with an alternative distribution of ACL.

#3

Thanks for such a quick reply.

I have build TVM with such config:

  • -DUSE_ARM_COMPUTE_LIB=OFF
  • -DUSE_ARM_COMPUTE_LIB_GRAPH_EXECUTOR=${ACL_PATH}

where ACL_PATH is the directory of arm_compute-v23.08-bin-android-arm64-v8a-neon-cl.tar.gz release package.

Then run test_arm_compute_lib/test_runtime.py on target mobile device (which has Mali G72) with RPC.

But it seems that running on cpu rather than Mali G72 (code).

How to make sure it is running on Mali GPU, thanks!

Here is cmake detail:

export ANDROID_NDK="/the/path/to/ndk"
export ACL_PATH="/the/path/to/acl"

cmake \
    -DCMAKE_SYSTEM_NAME=Linux \
    -DCMAKE_SYSTEM_PROCESSOR=aarch64 \
    -DCMAKE_CXX_COMPILER=${ANDROID_NDK}/toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android29-clang++ \
    -DCMAKE_C_COMPILER=${ANDROID_NDK}/toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android29-clang \
    -DUSE_OPENCL=ON \
    -DUSE_ARM_COMPUTE_LIB=OFF \
    -DUSE_ARM_COMPUTE_LIB_GRAPH_EXECUTOR=${ACL_PATH} \
    -DUSE_LIBBACKTRACE=OFF \
    -DUSE_PROFILER=ON \
    ..

make -j16 tvm_runtime
#4

But it seems that running on cpu rather than Mali G72 (code ).

How to make sure it is running on Mali GPU, thanks!

#5

Hi @xhliu, I’m afraid the ACL integration doesn’t currently support GPU, it will only offload operations to CPU.