Unable to reproduce benchmark on Mobile GPU rk3399

Troubleshooting
1

Hi,

I have been struggling with getting good results on Firefly Mali GPU board in the past few weeks.

For example, I have followed exact tutorial step by step (including turn-off lightdm and locked the GPU’s frequency to performance mode).

I did both auto-tuning tutorial and benchmark test listed here:

  1. https://docs.tvm.ai/tutorials/autotvm/tune_relay_mobile_gpu.html#sphx-glr-tutorials-autotvm-tune-relay-mobile-gpu-py
  2. https://github.com/apache/incubator-tvm/tree/master/apps/benchmark

Here are my results for benchmark test:

image
image990×644 76 KB

My tuning also did not go well. The tutorial claimed 120 ms ish, but I got 170ms ish.

2

@merrymercy Hi Lianmin, I saw you are mostly in charge of TVM support for rk3399 and mali gpu. I would really appreciate if you could help me look into this issue. The following screenshot is my benchmark measurement with Mali gpu. I have also observed that there is not that much difference in arm cpu benchmark. Thanks very much!

image
image948×308 60.6 KB

3

I didn’t maintain the benchmark for a long time. Currently, I don’t have a board for testing either. This reflects one big problem of our CI : we don’t do performance regression test.

It has been more than one year since the benchmark was released. A lot of changes has been done and I am not tracking them. You can try to go back to this commit (https://github.com/apache/incubator-tvm/pull/3131) or even the earlier one (https://github.com/apache/incubator-tvm/pull/2035). If you board is rk3399, you should be able to reproduce the benchmark by our pre-tuned schedule from TopHub, without tuning.

2 Likes
4

Thanks very much for your response!! I found there is a PR screwed up the mali performance. I converted it back to a PR on June 2019, and It worked now.

5

Maybe you can open an issue about your findings?