What is the current plan for embedded?

Questions
#1

Hi,

I tried to follow to current development around µTVM and AOT compilation, but still quite a bit confused, as it seems like the AOT will replace the efforts around µTVM’s runtime?

Will the µTVM runtime development be continued? If I understood correctly, according to the RFC for AOT there is also some runtime functionality required for it? And is Zephyr RTOS still required for it?

Thanks in advance

#2

@max1996 sorry to hear about your confusion. You have mentioned three concepts here which I try to distinguish them. The development of AOT is separate from microTVM and Zephyr RTOS.

  • AOT is a new runtime where the runtime doesn’t need to parse the graph json on the target and therefore can save a lot of memory.
  • microTVM could use the AOT or use the previous graph runtime. Using AOT allows us to fit larger models since we saved lots of memory by removing graph parser.
  • And then we have Zephyr RTOS. Currently, we use Zephyr RTOS since it is popular and easy to use for development. However, microTVM and AOT could potentially be implemented in a bare metal microcontroller or other RTOSs (Mbed OS, TI-RTOS, Arduino, …). That being said, most of our current developments and demos are based on Zephyr. For example:
    • AOT Demo implements an AOT demo for microTVM using ZephyrRTOS.
    • Host Driven implements a demo for microTVM using graph executor runtime again on ZephyrRTOS.

There are efforts on adding Project API feature which makes integration of any RTOS much smoother. Therefore, the development of AOT and microTVM will be generic and each RTOS will have its own project API development files in TVM.

Hope this makes it clear and happy to follow up.

1 Like
#3

cc @areusch to add more on this if I missed anything.

1 Like
#4

I wanted to try the AOT Demo, is there any example on how to actually run this demo ? I can not figure out on how exactly do we build the final executable for the MCU with the model.

#5

hi @ebad,

currently it’s tested in CI by the script tests/micro/zephyr/test_zephyr_aot.py. you can run it using this script. there is a lot of work ongoing right now on AOT and i wouldn’t say we are quite ready for a user-facing demo just yet. however, feel free to check out what we’ve got with that script. you’ll need to launch it using ci-qemu:

$ docker/bash.sh ci_qemu tests/scripts/task_config_build_qemu.sh
$ docker/bash.sh ci_qemu tests/scripts/task_build.sh build -j2
$ docker/bash.sh ci_qemu python3 tests/micro/zephyr/test_zephyr_aot.py

-Andrew

1 Like
#6

Hi @areusch . Until now, is there any AOT python example for user to run besides test_zephyr_aot.py file?

#7

@lumos there are also some test cases in https://github.com/apache/tvm/tree/main/tests/python/relay/aot/test_crt_aot.py