Inference time on PYNQ-Z1

Hi all,

I try to get a sense of the inference time for Resnet-18 on PYNQ-Z1. As I run the auto-tuning example on my PYNQ board, the fully-tuned version (grid search all schedules) takes 416 ms for the final inference with a single image, and a barely tuned version (grid search with only 1 trial) gives me just slight worse time of 432 ms. As some references, the sample output from the tutorial shows 621.79 ms, and the published VTA paper (Fig. 8) can achieve ~250 ms.

If I understand correctly, the TimeEvaluator function only measures the time on the board, so I’m trying to understand the variations behind these runs, and a reasonable number to expect for ResNet-18 on PYNQ-Z1. Below is a table that summarises the numbers above.

    Version                   Time  (std dev)  
    fully tuned (ours).       416.24 ms (5.42 ms)
    barely tuned (ours)       432.34 ms (41.63 ms)
    sample output             621.79 ms (0.14 ms)
    VTA paper                 ~250 ms

Any help is appreciated. Thanks!

@tqchen @thierry Apologies if not asking the right person, but any help is appreciated.