Performance drop after tuning resnet50 with Ansor

[VarunGupta]

I’m trying to tune resnet50 on Adreno GPU on QNX OS using Ansor. The tuning succeeds but the performance of tuned resnet50 models is way worse than untuned model. Following are the inference time of tuned and untuned models: Untuned Resnet50 fp32 onnx model : 42 ms Tuned Resnet50 fp32 model : 80 ms

I’ve tuned resnet for 20000 trials. The model was tuned via ansor in “NHWC” format. How to resolve this discrepancy? @srkreddy1238 @comaniac @tqchen

[varunnaw]

Are you using Ansor with the CLML extensions? I saw somewhere that CLML uses NCHW. I’m not totally sure, but I do know that the cpp_clml.py script in the tvm project (which converts models to a complete clml model with host code) turns the model into NCHW.

For me, on top off seeing a performance drop, the search process is very slow.

Ansor - number of tuning trials

I am tuning and compiling a model with a number of different targets (llvm, metal, opencl) and just came across this useful info about the number of tuning trials in this article: You can set it to a small number (e.g., 200) for a fast demonstrative run. In practice, we recommend setting it around 800 * len(tasks), which is typically enough for the search to converge. My model is quite large - it has ~72 tasks, which means the suggested number of trials would be 57600. A tuning run with 100…

@srkreddy1238 @comaniac @lianminzheng