Hi there,
I try to get a sense of the inference time for Resnet-18 on PYNQ-Z1. As I run the auto-tuning example on my PYNQ board, the fully-tuned version (grid search all schedules) takes 416 ms for the final inference with a single image, and a barely tuned version (grid search with only 1 trial) gives me just slight worse time of 432 ms. As some references, the sample output from the tutorial shows 621.79 ms, and the published VTA paper (see Fig. 8) can achieve ~250 ms.
If I understand correctly, the TimeEvaluator
function only measures the time on the board, so I’m trying to understand the variations behind these runs, and a reasonable number to expect for ResNet-18 on PYNQ-Z1. Below is a table that summarises the numbers above.
Version Time (std dev)
fully tuned (ours). 416.24 ms (5.42 ms)
barely tuned (ours) 432.34 ms (41.63 ms)
sample output 621.79 ms (0.14 ms)
VTA paper ~250 ms
Any help is appreciated. Thanks!
Zhan