So what happens is that both input_data.h and output_data.h files included from main.c currently need to be generated and provided by the user. Their content are, respectively, the input data and the output location for the model, so the match the input and output tensor types of the model.
If you take a look at src/model/codegen/host/include/tvmgen_default.h header you’ll find that there are definitions for two structs: struct tvmgen_default_inputs and tvmgen_default_outputs. Also, in this very same file, there is a definition for the tvmgen_default_run(), which has these two structs forming its signature.
tvmgen_default_run() is the entry point for calling the compiled model (in your case the cnn_class_channel_h_w_3_1_4_1.onnx model), hence a struct tvmgen_default_inputs must be defined defined (usually in main.c) and its member must point to the input data, that comes from “data_input.h”. The same happens for the output, regarding “output_data.h”.
Of course, you can also thrown out the #include "data_input.h" (and/or "output_data.h"and declare them directly in the main.c.
Once the input and output structs are declared properly, one just needs to call tvmgen_default_run() passing them to it and the magic happens (the model runs and the output will be put in the output struct).
Here is a very simple example I wrote about a month ago where you can see the input and output defined in the “input_data.h” and “output_data.h”: GitHub - gromero/sine_aot: Example of sine (sin()) inference model running using AOTExecutor on a STM32F746 disco board
Please note that I tweaked a bit the tvmgen_default_run() signature to pass an array directly (without using the structs). That was a quick hack, hence it should be avoided. Also it is a month ago example, so some names in the generated structs might have changed to match better the names in the model taken from the Relay module (I think it can change if you’re importing the model a different format, like tflite or something that doesn’t provide a meaningful name for the inputs).
There you can see the Python scripts which are called to generate inputs.h (equivalent to input_data.h in our discussion here) from a .jpg file for the mobilenet model.
@gromero, thanks for the answer.
I was able to build the project. However, when I execute the command to run the model, I get an error as follows:
tvmc run --device micro tvm/apps/microtvm/zephyr/template_project/project_cnn --print-top 6 --project-option zephyr_board=nucleo_f746zg zephyr_base=ZEPHYR_BASE openocd_serial=OPENOCD_SERIAL
Error: Could not open a session with the micro target.
@asnecemnnit did you specify the proper board serial number in place of OPENOCD_SERIAL? also, you might need to repeat --project-option before each option:
@areusch , even with following command, I get the same error:
tvmc run --device micro tvm/apps/microtvm/zephyr/template_project/project_cnn --print-top 6 --project-option zephyr_board=nucleo_f746zg
Error: Could not open a session with the micro target.
@asnecemnnit I think we need some more logs before it’s clear what’s happening here. We don’t have a good command-line flag for this now in tvmc (filed #10629). Can you try…
Hi @asnecemnnit Pardon, I’ve been a bit sick and so slow to respond.
Running an AOT model using tvmc run will never work on the current state of affairs. This is because tvmc run will try to open a RPC session and AOT runtime currently does not support it (I think it’s planned cc:ing @areusch@Mousius to shed some light here, I always get confused on what’s the plan for AOT + RPC, once graph executor will probably get replaced by the AOT executor). Hence under the hood of Error: Could not open a session with the micro target. you’re probably getting a:
tvm._ffi.base.TVMError: MicroSessionTimeoutError: session start handshake failed after 5s
BTW, tvmc does not expect multiple --project-option (so one before each option). Hence only the options after the last occurrence of --project-option are considered.
That said, if you’ve correctly provided the inputs, the outputs, and accordingly set the WORKSPACE and your AOT runtime is OK and flashed to the device (using tvmc micro build and tvmc micro flash) you can jump to a terminal (e.g. minicom) and proceed to execute the model from there. All you need is to input the following commands: “init%” and then “infer%”, then TVMInfer() will get called your model executed, for instance:
Welcome to minicom 2.8
OPTIONS: I18n
Port /dev/ttyACM1, 23:05:37
Press CTRL-A Z for help on special keys
TVM AOT runtime
wakeup
TVMInfer()
result:0:0
I’ve put some TVMLogf() there to print TVM AOT runtime in main() before entering the infinite loop so I can at least check if minicom is talking nicely to the device.
I recognize there are some really rough edges in this workflow of using tvmc {compile,micro, run} when the AOT executor/interface/runtime are selected. You’ve crossed with the two main ones: setting the inputs and outputs once a project is created and running / getting the results backs using tvmc run (which was primarily written and tested a lot with the Graph executor and is not supported for AOT executor).
@gromero thanks for the detailed explanation. It makes much sense now. I’ll try the minicom approach and come back if i face any issues. Thanks again!