TVMC (micro) AOT error

microTVM
#1

@gromero, @areusch

I am trying to compile, build, flash and run AOT-Demo microTVM project using TVM CLI (tvmc) as follows:

tvmc compile tvm/explored_models/cnn_class_channel_h_w_3_1_4_1.onnx --target "c -keys=cpu -march=armvv7e-m -mcpu=cortex-m7 -model=stm32f746xx" --runtime crt --runtime-crt-system-lib=0 --executor 'aot' --executor-aot-workspace-byte-alignment=1 --executor-aot-unpacked-api=1 --executor-aot-interface-api=1 --output cnn_test.tar --output-format mlf --pass-config tir.disable_vectorize=1 --disabled-pass="AlterOpLayout"

tvmc micro create --force ./tvm/apps/microtvm/zephyr/template_project/project_cnn ./cnn_test.tar zephyr --project-option project_type=aot_demo zephyr_board=nucleo_f746zg

tvmc micro build --force tvm/apps/microtvm/zephyr/template_project/project_cnn zephyr --project-option zephyr_board=nucleo_f746zg west_cmd=west

tvmc micro flash  tvm/apps/microtvm/zephyr/template_project/project_cnn zephyr --project-option zephyr_board=nucleo_f746zg

However, I am getting an error from “tvmc micro build command…” as follows:

[ 91%] Building C object modules/stm32/stm32cube/CMakeFiles/..__modules__hal__stm32__stm32cube.dir/stm32f7xx/drivers/src/stm32f7xx_ll_rng.c.obj
[ 92%] Building C object modules/stm32/stm32cube/CMakeFiles/..__modules__hal__stm32__stm32cube.dir/stm32f7xx/drivers/src/stm32f7xx_ll_utils.c.obj
[ 92%] Linking C static library lib..__modules__hal__stm32__stm32cube.a
/home/vagrant/tvm/apps/microtvm/zephyr/template_project/project_cnn/src/main.c:32:10: fatal error: input_data.h: No such file or directory
   32 | #include "input_data.h"
      |          ^~~~~~~~~~~~~~
compilation terminated.
CMakeFiles/app.dir/build.make:75: recipe for target 'CMakeFiles/app.dir/src/main.c.obj' failed
make[2]: *** [CMakeFiles/app.dir/src/main.c.obj] Error 1
CMakeFiles/Makefile2:2109: recipe for target 'CMakeFiles/app.dir/all' failed
make[1]: *** [CMakeFiles/app.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 92%] Built target ..__modules__hal__stm32__stm32cube
Makefile:90: recipe for target 'all' failed
make: *** [all] Error 2
The following error occured on the Project API server side:  calling method build: JSON-RPC error # -32000: calling method build
Traceback (most recent call last):
  File "/home/vagrant/tvm/python/tvm/micro/project_api/server.py", line 481, in serve_one_request  # <--- Outermost server-side stack frame
    self._dispatch_request(request)
  File "/home/vagrant/tvm/python/tvm/micro/project_api/server.py", line 593, in _dispatch_request
    return_value = dispatch_method(**params)
  File "/home/vagrant/tvm/apps/microtvm/zephyr/template_project/project_cnn/microtvm_api_server.py", line 515, in build
    check_call(args, cwd=BUILD_DIR)
  File "/home/vagrant/tvm/apps/microtvm/zephyr/template_project/project_cnn/microtvm_api_server.py", line 85, in check_call
    return subprocess.check_call(cmd_args, *args, **kwargs)
  File "/usr/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', '-j2']' returned non-zero exit status 2.

I would like to know if I am doing anything in a wrong way. And what can I do to resolve this build error?

#2

Hi,

So what happens is that both input_data.h and output_data.h files included from main.c currently need to be generated and provided by the user. Their content are, respectively, the input data and the output location for the model, so the match the input and output tensor types of the model.

If you take a look at src/model/codegen/host/include/tvmgen_default.h header you’ll find that there are definitions for two structs: struct tvmgen_default_inputs and tvmgen_default_outputs. Also, in this very same file, there is a definition for the tvmgen_default_run(), which has these two structs forming its signature.

tvmgen_default_run() is the entry point for calling the compiled model (in your case the cnn_class_channel_h_w_3_1_4_1.onnx model), hence a struct tvmgen_default_inputs must be defined defined (usually in main.c) and its member must point to the input data, that comes from “data_input.h”. The same happens for the output, regarding “output_data.h”.

Of course, you can also thrown out the #include "data_input.h" (and/or "output_data.h"and declare them directly in the main.c.

Once the input and output structs are declared properly, one just needs to call tvmgen_default_run() passing them to it and the magic happens (the model runs and the output will be put in the output struct).

Here is a very simple example I wrote about a month ago where you can see the input and output defined in the “input_data.h” and “output_data.h”: https://github.com/gromero/sine_aot Please note that I tweaked a bit the tvmgen_default_run() signature to pass an array directly (without using the structs). That was a quick hack, hence it should be avoided. Also it is a month ago example, so some names in the generated structs might have changed to match better the names in the model taken from the Relay module (I think it can change if you’re importing the model a different format, like tflite or something that doesn’t provide a meaningful name for the inputs).

A better example maybe can be found in the Ethos-U demo: https://github.com/apache/tvm/blob/main/apps/microtvm/ethosu/run_demo.sh#L169-L170

There you can see the Python scripts which are called to generate inputs.h (equivalent to input_data.h in our discussion here) from a .jpg file for the mobilenet model.

HTH.

1 Like
#3

@gromero, thanks for the answer. I was able to build the project. However, when I execute the command to run the model, I get an error as follows:

tvmc run --device micro tvm/apps/microtvm/zephyr/template_project/project_cnn --print-top 6 --project-option zephyr_board=nucleo_f746zg zephyr_base=ZEPHYR_BASE openocd_serial=OPENOCD_SERIAL
Error: Could not open a session with the micro target.

Do you suspect any reasons?

#4

@asnecemnnit did you specify the proper board serial number in place of OPENOCD_SERIAL? also, you might need to repeat --project-option before each option:

tvmc run --device micro tvm/apps/microtvm/zephyr/template_project/project_cnn --print-top 6 --project-option zephyr_board=nucleo_f746zg --project-option zephyr_base=ZEPHYR_BASE --project-option openocd_serial=OPENOCD_SERIAL
#5

@areusch , even with following command, I get the same error:

tvmc run --device micro tvm/apps/microtvm/zephyr/template_project/project_cnn --print-top 6 --project-option zephyr_board=nucleo_f746zg
Error: Could not open a session with the micro target.
#6

@asnecemnnit I think we need some more logs before it’s clear what’s happening here. We don’t have a good command-line flag for this now in tvmc (filed #10629). Can you try…

python3 -c 'import logging; import tvm.driver.tvmc.main; logging.getLogger().setLevel(logging.DEBUG); tvm.driver.tvmc.main.main()' run --device micro tvm/apps/microtvm/zephyr/template_project/project_cnn --print-top 6 --project-option zephyr_board=nucleo_f746zg

I think this should give you some more logs which we can use to troubleshoot why it can’t find your board.

#7

@areusch it still returns

Error: Could not open a session with the micro target.
#8

Hi @asnecemnnit Pardon, I’ve been a bit sick and so slow to respond.

Running an AOT model using tvmc run will never work on the current state of affairs. This is because tvmc run will try to open a RPC session and AOT runtime currently does not support it (I think it’s planned cc:ing @areusch @Mousius to shed some light here, I always get confused on what’s the plan for AOT + RPC, once graph executor will probably get replaced by the AOT executor). Hence under the hood of Error: Could not open a session with the micro target. you’re probably getting a:

tvm._ffi.base.TVMError: MicroSessionTimeoutError: session start handshake failed after 5s

BTW, tvmc does not expect multiple --project-option (so one before each option). Hence only the options after the last occurrence of --project-option are considered.

That said, if you’ve correctly provided the inputs, the outputs, and accordingly set the WORKSPACE and your AOT runtime is OK and flashed to the device (using tvmc micro build and tvmc micro flash) you can jump to a terminal (e.g. minicom) and proceed to execute the model from there. All you need is to input the following commands: “init%” and then “infer%”, then TVMInfer() will get called your model executed, for instance:

Welcome to minicom 2.8

OPTIONS: I18n                                                                
Port /dev/ttyACM1, 23:05:37                                                  
                                                                             
Press CTRL-A Z for help on special keys                                      
                                                                             
TVM AOT runtime                                                              
               wakeup                                                        
                     TVMInfer()                                              
                               result:0:0

I’ve put some TVMLogf() there to print TVM AOT runtime in main() before entering the infinite loop so I can at least check if minicom is talking nicely to the device.

I recognize there are some really rough edges in this workflow of using tvmc {compile,micro, run} when the AOT executor/interface/runtime are selected. You’ve crossed with the two main ones: setting the inputs and outputs once a project is created and running / getting the results backs using tvmc run (which was primarily written and tested a lot with the Graph executor and is not supported for AOT executor).

#9

@gromero thanks for the detailed explanation. It makes much sense now. I’ll try the minicom approach and come back if i face any issues. Thanks again!

#10

Did minicom approach work for you @asnecemnnit ? I tried as mentioned in this thread with init%, infer% in picocom. It is not working.

#11

@videvelop, as far as I remember, it worked for me after a few tries. Sadly, I can’t help you as I am not working on this project anymore.