Error when trying to compile the pre-quantized pytorch model

Troubleshooting
#1

Hi Community, I am following tutorial: Deploy a Framework-prequantized Model with TVM — tvm 0.9.dev0 documentation to build a pre-quantized mobilenet v2 model. But it throws an error:

Traceback (most recent call last): File “pytorch_test_quant.py”, line 76, in test_compile(model_config) File “pytorch_test_quant.py”, line 60, in test_compile mod, params = relay.frontend.from_pytorch(script_module, input_shapes) File “/home/.local/lib/python3.8/site-packages/tvm-0.8.0-py3.8-linux-x86_64.egg/tvm/relay/frontend/pytorch.py”, line 3882, in from_pytorch input_scales_for_bias = qnn_torch.add_input_quant_params_to_op_inputs(graph) File “/home/.local/lib/python3.8/site-packages/tvm-0.8.0-py3.8-linux-x86_64.egg/tvm/relay/frontend/qnn_torch.py”, line 475, in add_input_quant_params_to_op_inputs scale, zp = _get_quant_param_for_input(node.inputsAt(i)) File “/home/.local/lib/python3.8/site-packages/tvm-0.8.0-py3.8-linux-x86_64.egg/tvm/relay/frontend/qnn_torch.py”, line 298, in _get_quant_param_for_input return dfs(input_value.node()) File “/home/.local/lib/python3.8/site-packages/tvm-0.8.0-py3.8-linux-x86_64.egg/tvm/relay/frontend/qnn_torch.py”, line 286, in dfs scale = current_node.inputsAt(indices[0]) RuntimeError: ArrayRef: invalid index Index = 6; Length = 6

Anyone comes across the same issue?

#2

OK. I think it is caused by pytorch version.

If using torch==1.10.1 and torchvision==0.11.2, it will have this issue, and it will even fail the test: https://github.com/apache/tvm/blob/main/tests/python/frontend/pytorch/qnn_test.py

If I roll back to torch==1.7.0 and torchvision==0.8.1, the test works.

I saw the commit of upgrading torch to 1.10.1 in this commit: [Docker] Update to Torch 1.10.1 (#9781) · apache/tvm@b29a443 · GitHub

I wonder why the test for this commit has not failed.

Any comments? @masahi

Thanks!

#3

Probably your TVM is not up-to-date. The test runs on CI so it is supposed to work.

#4

Got it. Thanks! I think I am on v0.8 official release, which probably did not include your change from last Dec.

#5

I also experienced this issue and found that rolling back PyTorch helped, as an earlier comment said.

I looked at the versions used in the CI pipeline for v0.8 (which is what I’m using here). This can be a helpful heuristic for similar issues.

[quantization] Performance degradation of quantization