I was trying to run different models in Relay and there is one issue with the executor.evaluate() API. TVM will only take the input with a shape exactly like the input shape of the model, instead of a smaller batch size. And that is commonly acceptable on other frameworks like Keras or PyTorch. For example, instead of doing:
tvm_out = executor.evaluate()(tvm.nd.array(input_data), **params)
can we do
tvm_out = executor.evaluate()(tvm.nd.array(input_data[:1000]), **params)
in TVM?