Low TVM performance with Galaxy S20+(CPU)

Questions
1

image

Hi,

I’ve tested VGG-16 model on Galaxy S9 and Galaxy S20+. First, I’ve just converted VGG-16 pytorch model to tflite model and measured fps and accuracy. As I expected, Galaxy S20+(CPU) shows higher fps than that of Galaxy S9(CPU). However, when I tried to do TVM build the VGG-16 pytorch model and measured fps and accuracy using TVM RPC app (cf. https://tvm.apache.org/docs/tutorials/frontend/deploy_model_on_android.html?highlight=docker) Galaxy S9(CPU) shows higher fps than that of Galaxy S20+(CPU). The same tendency was shown in TVM+autotune. I’ve used the following target for both.

arch = “arm64”
target = tvm.target.Target(“llvm -mtriple=%s-linux-android” % arch)

Do you know the reason for this situation? Could you give me some advice?

Thanks!

2

Maybe cpu is at low energy mode ?

###### arm linux  ######
➜  ~ cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
performance
performance
performance
performance
➜  ~
Connection to 192.168.31.79 closed.


###### rk3399 Android ########
➜  Playground adb shell
* daemon not running; starting now at tcp:5037
* daemon started successfully
126|rk3399_all:/ $ cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
interactive
interactive
interactive
interactive
interactive
interactive
rk3399_all:/ $

Do some warmup before benchmark.

3

Thank you for your reply. What is the exact meaning of warmup? 1) does it mean that I should turn on the phone screen for a certain period of time (e.g., 15 minutes) and then take measurements? 2) Do I need to revise the source code to avoid the warmup period?

4

Warm up means that you need to run the model before do the measurements.