[keras] Why Input with shape (1, 2, 2) is not valid for operator Dense?

Questions
#1

Why shape (1, 2, 2) is not valid input for Dense in TVM? From keras documentation, no such limitation was found.

Crash message

Traceback (most recent call last):
  File "test.py", line 20, in <module>
    mod, params = relay.frontend.from_keras(model, {'input_1': input_shape})
  File "/workplace/software/tvm/tvm_/python/tvm/relay/frontend/keras.py", line 1540, in from_keras
    _convert_layer(keras_layer, etab)
  File "/workplace/software/tvm/tvm_/python/tvm/relay/frontend/keras.py", line 1494, in _convert_layer
    layout,
  File "/workplace/software/tvm/tvm_/python/tvm/relay/frontend/keras.py", line 1373, in keras_op_to_relay
    outs = _convert_map[op_name](inexpr, keras_layer, etab, data_layout)
  File "/workplace/software/tvm/tvm_/python/tvm/relay/frontend/keras.py", line 265, in _convert_dense
    "Input shape {} is not valid for operator Dense.".format(input_shape)
tvm.error.OpAttributeInvalid: Input shape (1, 2, 2) is not valid for operator Dense.

reproducable script

import tvm.relay as relay
import numpy as np
from tensorflow import keras
from tensorflow.keras import layers, models


input_shape = (1, 2, 2)
input_data = np.random.random(input_shape)
x = layers.Input(shape=(2, 2), dtype='float32')

layer = keras.layers.Dense(units=3)
layer.set_weights(layer.get_weights())

y = layer(x)
model = models.Model(x, y)
print(model.summary())

mod, params = relay.frontend.from_keras(model, {'input_1': input_shape})
#2

The crash location is shown below:

image
image810×450 28.8 KB

Why set the limitation: if input_dim >2, the second dimension must be ‘1’?

#3

This is probably only because our dense op expects 2D input. What other frontends do for > 2D input is to simply reshape all “batch” axes into one. In your case, (1, 2, 2) → (1 * 2, 2).

2 Likes
#4

@masahi Thanks for your valuable explanation. If we need to modify the dense in Keras frontend to reshape dim>2D into 2D as the other frontend? If so, I’m willing to submit a pr to fix it.

#5

Yes, you are welcome to do so.