Hi,
I’m new to TVM and Relay and I’m trying to figure out how a torch model is transferred to Relay. So I tried some neural network layers and to check what will the Relay module be like. And I found something really interesting. The pyTorch code I used is like the following:
model = torch.nn.Sequential(
torch.nn.Linear(3, 1),
)
....
shape_list = [("input0", (5,3))]
mod, params = relay.frontend.from_pytorch(script_model, shape_list)
Then I got the output Relay module like:
def @main(%input0: Tensor[(5, 3), float32], %v0.weight: Tensor[(1, 3), float32], %v0.bias: Tensor[(1),
float32]) {
%0 = transpose(%v0.weight, axes=[1, 0]);
%1 = transpose(%0, axes=[1, 0]);
%2 = nn.dense(%input0, %1, units=1);
add(%2, %v0.bias)
}
As you can see the two transpose operations cancel each other. I’m wondering how they are generated and will they affect the running efficiency?. (ps. this doesn’t happen if the module is transferred from Keras)
I would really appreciate it if someone gives me any explanations.