Hi @joyalbin,
Unfortunately I never tried this scenario, and I am not Android expert. I think you have two possibilities:
- Running the RPC server on Android
- Statically compiling every together, copy the binary on Android and run
RPC on android
In theory it should be similar to the way you use the RPC server on a linux board (like the Raspi).
You should follow those instructions:
- https://tvm.apache.org/docs/deploy/android.html
- https://tvm.apache.org/docs/tutorials/frontend/deploy_model_on_android.html
- https://github.com/apache/incubator-tvm/tree/main/apps/android_rpc
And instead of USE_OPENCL you should set USE_ARM_COMPUTE_LIB_GRAPH_RUNTIME to the Arm Compute Library path. Hopefully, this will statically link against the library, so that your RPC server on the phone will have all you need.
Statically linked binary
If that doesn’t work, another solution would be to deploy the model as a stand-alone. This is the guide: https://github.com/apache/incubator-tvm/tree/main/apps/bundle_deploy
The idea is to follow bundle_static and statically link everything together (runtime+library), so that you can copy a single binary to your phone and execute the network.
This option is easier, but it won’t let you auto-tune the network (for that you need to send different workloads to the board)
Hope this helps