Help！Error when getting starting using TVMC

#1

My code is here:

from tvm.driver import tvmc

# Step 1: Load a model
model = tvmc.load('my_model.onnx')
# now the model is in Relay

# Step 2: Compile
# translates the model from Relay to a lower-level language
package = tvmc.compile(model, target='llvm')

# Step 3: Run
result = tvmc.run(package, device='cpu')
print(result)

An error occurred:

Check failed: (pval != nullptr) is false: Cannot allocate memory symbolic tensor shape [?, 3, 224, 224]

The model is OK (resnet50 used in tutorial), what can I do?

#2

Then I change a model(handwriten number prediction) trained by myself, it works and returns:

One or more operators have not been tuned. Please tune your model for better performance. Use DEBUG logging level to see more details.
2022-08-31 11:42:08.325 INFO load_module /tmp/tmpjfn7p5vb/mod.so
Execution time summary:
 mean (ms)   median (ms)    max (ms)     min (ms)     std (ms)

   0.1011       0.1009       0.1088       0.0979       0.0028


Output Names:
 ['output_0']

Obvisiously, the model resnet50 in tutorial is not OK. But why?

#3

I think your model file my_model.onnx uses batch size as a symbolic constant '?' for the input tensor. Updating the model’s batch size with some constant might work. Use https://netron.app/ to visualize.

#4

Thanks, you are right.
The key point is that tvm tutorial using TVMC Python is misleading. I just follow the tutorial step by step.

#5

@denis the error message you’re seeing is often caused by using a model that has variable shape size.

Tracking down the changes on ONNX repository, there was this change which changed the models to have variable batch size: https://github.com/onnx/models/pull/537

To use models like this in TVMC both CLI and API, you’ll need to provide the input shape, as documented in the command line tutorial: Compiling and Optimizing a Model with TVMC — tvm 0.10.dev0 documentation

So your fix might be just calling model = tvmc.load('my_model.onnx', shape_dict={'data' : [1,3,224,224]}), to give the shape information as expected (I didn’t test this suggestion, but the main idea is that a shape size will need to be provided).

The key point actually is that tutorials, as with any piece of code, might have bugs or get outdated by external changes, and are not intentionally misleading.

With that in mind, I strongly encourage you, when finding these issues, to report them using the issue reporting tool at https://github.com/apache/tvm/issues/new/choose

1 Like
#6

Hi @leandron ,
Thanks a lot! :heart: This is the best answer I’ve recived. :100: