I’m trying to understand current performance on Intel targets. I’m just wondering if you have any benchmarks for Intel CPU/GPU and some benchmarking results for such targets? There are some posted for ARM, Mali and NVIDIA already, but I didn’t see anything for Intel.
there is one I did in last Oct. here
The input size is (1, 3, 224, 224) and all numbers are in millisecond.
2 Likes
Thanks! I’ll use you numbers to reproduce those on my setup and use them as reference. Am I right that models were taken from gluon model zoo as for most of the tutorials in TVM repo? Did you have CPU frequency fixed or leave it as is? Did you have any GPU numbers so far? Did you use python or C API? Did you use stream API?
We are still working on benchmarking the Intel GPUs results. Here is our benchmarking results of Intel CPUs presented in a preprint (check Table 2a on page 10).
1 Like
Thanks! I saw this paper, but it’s still not clear for me now there to find the exact models and exact TVM setup to reproduce those numbers. Could you please advice me on models (this model zoo should I use), HW setup (number of threads, freq, etc) and the benchmark itself (TMV parameters, c++/python). It would be great if you can point me to the benchmark itself. I may to extend it for GPU.
You can follow this tutorial to benchmark mxnet gluon model zoo cnn models: https://docs.tvm.ai/tutorials/nnvm/from_mxnet.html. The HW used in paper is c5.9xlarge with 18 physical cores. You need to set tvm target as “llvm -mcpu=skylake-avx512”. This will use default schedule and gives fair performance. Graph tuning is in PR and after it is merged the full performance can be reproduced.
1 Like
code related to the work of Optimizing CNN Model Inference on CPUs already merged into tvm? thanks
I’m looking for benchmarks for Intel Xe GPU (96 EU) Could anyone point me to those? Thanks for any help!