What should I input for device argmument when tuning models for android phone

#1

Hi, I have been looking at the tutorial of tvm to compile mobilenetv2 for android phone(arm cpu). But the tutorial is different in these to webset. For the first one, it uses -device arm-cpu. And when I run this code. I gives me a warning.

https://tvm.apache.org/docs/how_to/tune_with_autotvm/tune_relay_arm.html#sphx-glr-how-to-tune-with-autotvm-tune-relay-arm-py:~:text=target%20%3D%20tvm.target.Target("llvm%20-device%3Darm_cpu%20-mtriple%3Daarch64-linux-gnu")

For the second, it omits this argument. But the speed of inference is not updated.

https://tvm.apache.org/docs/how_to/tune_with_autoscheduler/tune_network_arm.html#sphx-glr-how-to-tune-with-autoscheduler-tune-network-arm-py:~:text=target%20%3D%20tvm.target.Target("llvm%20-mtriple%3Daarch64-linux-gnu%20-mattr%3D%2Bneon")

#2

These two tutorials use different tuning mechanisms, the first one uses AutoTVM, the second one Autoscheduler. If you use AutoTVM, you need to have the -device arm_cpu option present since that tells the tuner to use Arm specific TOPI schedules.

As the second tutorial says, Autoscheduler doesn’t make use of TOPI schedule templates (it only uses compute definitions), so you don’t need to add that option to the target string.

Usually AutoTVM performs better on Arm targets, but that really depends on a specific hardware and the specific device, so it’s probably good to try both. You might also want to explore if you can use other -mattr options in addition to +neon, eg +dotprod (that will again depend on the device), in case of AutoTVM that can make a difference in terms of which schedules are chosen :slight_smile:

1 Like
#3

I see. Thanks a lot.