Reshapes when importing tflite model

Questions
#1

Hi,

I am using TVM to generate C code for a Cortex-M device. The model input is the anomaly_detection tflite model from mlperf tiny benchmark that only contains a series of fully connected layers.

When importing the model I notice that TVM inserts reshape operators on every fully connected input. Why does the tflite importer insert a reshape operation on the input expression before each FullyConnected Operator?

Most of the reshapes operators are removed later, but which pass is removing the reshapes?

I also notice that in my case there is one reshape remaining after the relay is lowered. The remaining reshape is the first reshape in the graph, and I am trying to figure out how to remove this one as well in a clean way.

#2

I found the pass that removes the reshape operators. In my use case this is done inside the Codegen() function inside the AOTExecutorCodegen class. This function will use the pass called RemoveStandaloneReshapes(). This pass seems to remove all but the first reshape in my use case.

#3

Hi, @silabs-kjetil I am working in this field.

Here, in /tvm-path/python/tvm/relay/frontend/tflite.py
You could find following codes:

# Input shape: [i_batch_size, ..., n_inputs]
# Filter shape: [n_inputs, n_units]
#
# As we will transform Fully_Connected Input to Dense Op inputs as below
# Dense expected Input shape: [batch_size, n_units]
# Dense expected Weight shape: [out_dim, n_units]
# Dense output shape: [batch_size, out_dim]
target_shape = tuple((-1, weight_tensor_shape[1]))
in_expr = self.get_tensor_expr(input_tensor)
in_expr = _op.reshape(in_expr, target_shape)

The code _op.reshape insert reshape op.
If you want to remove reshape ops in your Relay IR converted from the anomaly_detection tflite model, just comment out the codes. I have tried it and this method works.
By the way, I don’t know why TVM developer inserts reshape operators on every fully connected input. In fully connected layer, Input shape[1] is always equal to Weight shape[1]. I think there is no need to insert reshape ops.