Hi TVM community,
I’m trying to deploy a tflite model which uses the layer UnidirectionalSequenceLSTM with TVM.
Doing some comparisons between tflite interpreter vs tvm runtime, I get a MAE in the range of 1e-2 (using random input data between [0, 1]). When, I reset the tflite model with reset_all_variables() between each inputs, the MAE gets down to the range of 1e-8.
I suspect the high difference is due to the tvm graph not maintaining statically the internal state of the LSTM layer.
Do you know if it’s a known limitation ? Is there any potential workaround ?
I could extract additional outputs and feed them back as inputs of the model to “manually” update the internal states…but it’s not easy to scale with this approach.
Thanks for your help, here is the code snippet to run the model through tvm runtime, please let me know if I’m missing something.
shape_dict = {"input_0": list(tf_model.input.shape)}
tvm_model, params = relay.frontend.from_tflite(tflite_model, shape_dict)
## compile the tvm model
with tvm.transform.PassContext(opt_level=3):
lib = relay.build(tvm_model, target="llvm", params=params)
tvm_runtime = graph_executor.GraphModule(lib["default"](tvm.cpu(0)))
for ind_data, data in enumerate(input_data):
tvm_runtime.set_input(
"serving_default_input_0:0",
np.expand_dims(data, 0),
)
tvm_runtime.run()