Trying to put an ONNX model through relay.frontend.from_onnx to get an IRModule results in
the following error:
The Relay type checker is unable to show the following types match.
In particular dimension 0 conflicts: 1 does not match 96.
The Relay type checker is unable to show the following types match.
In particular `Tensor[(96), float32]` does not match `Tensor[(1), float32]`
This error was encountered with the quantized ONNX CaffeNet-int8 model that can be
found here: https://github.com/onnx/models/tree/master/vision/classification/caffenet
The reference to Tensor[(96), float32] is most likely pointing to w_scale and
the subsequently generated requantize_scale since their TensorType is of the value
TensorType([96], float32).
Yeah it’s probably an issue with handling per-channel quantization correctly. I’ll have time later in the week or early next week to take a look at the problem.
Probably better to test on end to end Q models with per-channel Q, I’ve never seen such tests for quantized onnx model (only unit tests from onnx which doesn’t have good coverage).
What should have initially caught my eye was the axis=0 parameter to the requantize operator
outlined in the first post
which effectively means that it assumes the first axis to be the channel axis, since
axis is The channel axis for quantization. as per the docstring.
The following may be of some help:
I’ve tried to set axis=1 (though the channel axis should be somehow queried),
since ONNX assumes channel-first layout to be the default,
for the requantize mentioned above in addition to all of the quantize/dequantize/requantize
calls in the QuantizeLinear, DequantizeLinear and QLinearConv and tested with that.
With the above, trying to parse the models listed below that can be obtained from the ONNX
model zoo where the aforementioned CaffeNet model was taken from results in the following:
AlexNet - FAILED
The Relay type checker is unable to show the following types match.
In particular dimension 0 conflicts: 12288 does not match 256.
The Relay type checker is unable to show the following types match.
In particular `Tensor[(256), float32]` does not match `Tensor[(12288), float32]`
ResNet - PASSED
GoogleNet - PASSED
SqueezeNet - PASSED
ZFNet-512 - PASSED
ShuffleNet - PASSED
CaffeNet - FAILED
The Relay type checker is unable to show the following types match.
In particular dimension 0 conflicts: 12288 does not match 256.
The Relay type checker is unable to show the following types match.
In particular `Tensor[(256), float32]` does not match `Tensor[(12288), float32]`
VGG - PASSED
Otherwise all of the models mentioned produce an error similar to the one mentioned above.
EDIT: Adding direct reference to the quantize/dequantize/requantize calls changed in code: