I am testing the performance of the tvm model. When the qps is kept small until the cpu is full, the performance of the model will gradually decrease. Then adjust the qps to the lowest, the model performance cannot be restored to the same qps performance at the beginning of the test. Does anyone know this problem?
runtime config:
OMP_NUM_THREADS = 8
TVM_NUM_THREADS = 8