How to use Arm compute library with android RPC?

Dear All,

This is a continuation from the issue https://github.com/apache/incubator-tvm/issues/6857

@giuseros, Thank you for the detailed explanation. It got clear on the usage of the flags USE_ARM_COMPUTE_LIB_GRAPH_RUNTIME and USE_ARM_COMPUTE_LIB while compiling TVM

One thing still unclear is, how to use ARM compute library for android deployment? Please share me some example or document on the same?

I could see the flag USE_ARM_COMPUTE_LIB_GRAPH_RUNTIME is controlled only in the cmake, how can we use it for cross compilation or for Android RPC compilation?

How these two use cases in TVM can use ARM compute library.

Please someone help me in understanding…

Thank You

Hi @joyalbin,

Unfortunately I never tried this scenario, and I am not Android expert. I think you have two possibilities:

  • Running the RPC server on Android
  • Statically compiling every together, copy the binary on Android and run

RPC on android

In theory it should be similar to the way you use the RPC server on a linux board (like the Raspi).

You should follow those instructions:

And instead of USE_OPENCL you should set USE_ARM_COMPUTE_LIB_GRAPH_RUNTIME to the Arm Compute Library path. Hopefully, this will statically link against the library, so that your RPC server on the phone will have all you need.

Statically linked binary

If that doesn’t work, another solution would be to deploy the model as a stand-alone. This is the guide: https://github.com/apache/incubator-tvm/tree/main/apps/bundle_deploy

The idea is to follow bundle_static and statically link everything together (runtime+library), so that you can copy a single binary to your phone and execute the network.

This option is easier, but it won’t let you auto-tune the network (for that you need to send different workloads to the board)

Hope this helps

1 Like

@giuseros this was helpful… I got some idea to move forward.

Dear All, please share any performance comparison between TVM kernel and ACL kernel on any ARM devices (preferably android)?