[microTVM] Can't run pre-quantized TFLite model on NUCLEO board

[dongz]

I tried to follow the tutorial here, and I could finish it without error. But when I change the demo file from

to my own tflite, it can be built and flash to the board, but encountered errors in the open session,

And the error is:

I tried follow another tutorial about autotune with microTVM here, I could clean with that demo model too, but if use my testing tflite file, tasks = tvm.autotvm.task.extract_from_program(relay_mod["main"], {}, TARGET) ‘s return value tasks’ length is 0.

If I replace the model to TVM Conf 20 demo model, which in here, it can extract tasks, autotune, but running with MicroSessionTimeoutError: failed to read reply message after timeout 5s

My testing tflite files in here, mcunet folder, through jobs/download script.

Relay model logs, partial:

Relay build warnings:

Build config:

Environment:

macOS 12.3.1 + VirtualBOX 6.1.24 + Vagrant 2.7 + MacBook Pro 2018 15-inch(intel i7-8850H) + tutorial VM

I can pass the 9/10 zephyr test.

I’m a newbie to TVM microTVM and MCU, my board is nucleo_f767zi, I changed the board.json and target.py to support this model. I was very confused of this situation, hope for useful answers, thank you all.

[areusch]

Hi @dongz, thanks for trying out microTVM and apologies for the delay! I think what’s happening here is there isn’t enough RAM available on-device for microTVM to run your model via the host-driven Graph Executor. With that route, our error messaging is unfortunately not that great right now.

However, there are a couple of developments here which could allow you to proceed.

• You could try the standalone deploy flow, which runs the model on-device without requiring a host PC. We don’t have a great tutorial for this now–I’d suggest to try deploying standalone. To do that, try setting project_type to aot_demo in the TFLite tutorial.
• @alanmacd is working on building a host-driven version of aot_demo which would allow you to exercise the model from the command-line.

We’ll post up about this as these developments continue.

[dongz]

Thanks for your detailed reply, it helps a lot. Which mean I can’t auto tune either, if host-driven mode failed? Actually, I’m not sure this is a memory problem, cause this model only requires 70KB memory using STM32CUBE.ai runtime and the nucleo_h743zi has 1MB SRAM 2MB FLASH. I create a issue here, this problem can be reproduced by just replaced model to my test model, in here.

I really appreciate your help, and I look forward to your reply，

And when I set the target is ‘host’, I will encounter this error too.

[dongz]

It seemed like the aot_demo mode hasn’t finished, lacking some header file, like tvmgen_default.h. I found the input/output generate scripts for input/output in TVM Conf21 folder. @alanmacd @areusch

[dongz]

I found the tutorial seems to have some bugs about aot_mode, like codegen dir in mlf in tutorial:

but test cases in /path-of-tvm/tests/micro/zephyr/ is:

[dongz]

And I still can’t run my model in aot mode, facing a timeout error. I can pass the original aot test, but for mcunet in here encountered IOTimeout Error, which log in [https://pastebin.ubuntu.com/p/WCq3zSFh8q/plain/]. Even use nucleo_h743zi 1MB SRAM and 2MB Flash.