I’ve got a network with nothing but a single fully connected layer.
This should just be an
nn.dense(%input_1, %fc.weight, units=units)
However, looking at the generated IR, it actually generates:
%0 = transpose(%fc.weight, axes=[1, 0]) %1 = transpose(%0, axes=[1, 0]) %2 = nn.dense(%input_1, %1, units=units)
These transposes are redundant. They are these even when I apply the
ddo.simplify_fc_transpose.convert() Relay transformation on the module.
As a result, when I run
search_dense_op_weight, it returns nothing because the weight of
nn.dense is derived from other operations (at least, I think that’s why there’s nothing).
Any idea why these two redundant transposes are here, and why they are not being simplified?
EDIT: I’ve found that exporting the model to ONNX, and then importing into TVM bypasses the problem. So I think it’s an issue with the PyTorch to Relay importer.