Relay applies redundant double transpose to weight matrix

I’ve got a network with nothing but a single fully connected layer.

This should just be an nn.dense(%input_1, %fc.weight, units=units)

However, looking at the generated IR, it actually generates:

%0 = transpose(%fc.weight, axes=[1, 0]) 
%1 = transpose(%0, axes=[1, 0]) 
%2 = nn.dense(%input_1, %1, units=units) 

These transposes are redundant. They are these even when I apply the ddo.simplify_fc_transpose.convert() Relay transformation on the module.

As a result, when I run search_dense_op_weight, it returns nothing because the weight of nn.dense is derived from other operations (at least, I think that’s why there’s nothing).

Any idea why these two redundant transposes are here, and why they are not being simplified?

You can find all of the code to reproduce here.

EDIT: I’ve found that exporting the model to ONNX, and then importing into TVM bypasses the problem. So I think it’s an issue with the PyTorch to Relay importer.