Failed to run microtvm autotune example in ESP32

I tried to run microtvm autotune example (https://tvm.apache.org/docs/how_to/work_with_microtvm/micro_autotune.html) on ESP32, but stuck by the error:

with tvm.micro.Session(project.transport()) as session: File “/home/zgs/github/tvm/python/tvm/micro/session.py”, line 128, in enter _rpc_connect( File “/home/zgs/github/tvm/python/tvm/_ffi/_ctypes/packed_func.py”, line 237, in call raise get_last_ffi_error() tvm._ffi.base.TVMError: MicroSessionTimeoutError: session start handshake failed after 5s

Below is my steps with some work-around:

1. Add board info in file

~/github/tvm/build/microtvm_template_projects/zephyr/boards.json:

"esp32": {
"board": "esp32",
"model": "esp32",
"is_qemu": false,
"fpu": false,
"vid_hex": "",
"pid_hex": ""
}

2. Fix zephyr_base in the example code:

module_loader = tvm.micro.AutoTvmModuleLoader( template_project_dir=pathlib.Path(tvm.micro.get_microtvm_template_projects(“zephyr”)), project_options={ “zephyr_board”: BOARD, “west_cmd”: “west”, “verbose”: False, “project_type”: “host_driven”, “zephyr_base”: “/home/zgs/github/demo/ai/microtvm/zephyrproject/zephyr”, }, )

3. Checkout to zephyr v2.7-branch and install zephyr-sdk-0.13.1

wget https://github.com/zephyrproject-rtos/sdk-ng/releases/download/v0.13.1/zephyr-sdk-0.13.1-linux-x86_64-setup.run chmod +x zephyr-sdk-0.13.1-linux-x86_64-setup.run ./zephyr-sdk-0.13.1-linux-x86_64-setup.run – -d ~/zephyr-sdk-0.13.1 cd ~/zephyrproject/zephyr git checkout v2.7-branch cd … && west update export ZEPHYR_BASE=~/zephyrproject/zephyr

4. Work-around for esp32 compiled error with Zephyr

Due to the compile error:

/home/zgs/github/demo/ai/microtvm/zephyrproject/zephyr/include/posix/time.h:90:15: error: static declaration of ‘clock_gettime’ follows non-static declaration

__syscall int clock_gettime(clockid_t clock_id, struct timespec *ts);

^~~~~~~~~~~~~

In file included from /home/zgs/github/demo/ai/microtvm/zephyrproject/zephyr/include/posix/time.h:12,

from /home/zgs/github/demo/ai/microtvm/zephyrproject/zephyr/lib/posix/pthread_common.c:10:

/home/zgs/.espressif/tools/zephyr/xtensa-esp32-elf/xtensa-esp32-elf/sys-include/time.h:187:5: note: previous declaration of ‘clock_gettime’ was here

int clock_gettime (clockid_t clock_id, struct timespec *tp);

^~~~~~~~~~~~~

In file included from /home/zgs/github/demo/ai/microtvm/zephyrproject/zephyr/lib/posix/pthread_common.c:10:

/home/zgs/github/demo/ai/microtvm/zephyrproject/zephyr/include/posix/time.h:94:5: error: conflicting types for ‘timer_create’

int timer_create(clockid_t clockId, struct sigevent *evp, timer_t *timerid);

^~~~~~~~~~~~

In file included from /home/zgs/github/demo/ai/microtvm/zephyrproject/zephyr/include/posix/time.h:12,

from /home/zgs/github/demo/ai/microtvm/zephyrproject/zephyr/lib/posix/pthread_common.c:10:

/home/zgs/.espressif/tools/zephyr/xtensa-esp32-elf/xtensa-esp32-elf/sys-include/time.h:192:5: note: previous declaration of ‘timer_create’ was here

int timer_create (clockid_t clock_id,

^~~~~~~~~~~~

make[2]: *** [zephyr/lib/posix/CMakeFiles/lib__posix.dir/build.make:76: zephyr/lib/posix/CMakeFiles/lib__posix.dir/pthread_common.c.obj] Error 1

make[1]: *** [CMakeFiles/Makefile2:2950: zephyr/lib/posix/CMakeFiles/lib__posix.dir/all] Error 2

make[1]: *** Waiting for unfinished jobs…

I modify ~/github/demo/ai/microtvm/zephyrproject/zephyr/include/posix/time.h as follow to work-around the error:

5. Link error

/home/zgs/.espressif/tools/zephyr/xtensa-esp32-elf/bin/…/lib/gcc/xtensa-esp32-elf/8.4.0/…/…/…/…/xtensa-esp32-elf/bin/ld.bfd: /home/zgs/.espressif/tools/zephyr/xten sa-esp32-elf/xtensa-esp32-elf/lib/./libc.a(lib_a-locale.o): in function __locale_ctype_ptr_l’: /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/src/newlib/newlib/libc/locale/locale.c:983: undefined reference to __getreent’ /home/zgs/.espressif/tools/zephyr/xtensa-esp32-elf/bin/…/lib/gcc/xtensa-esp32-elf/8.4.0/…/…/…/…/xtensa-esp32-elf/bin/ld.bfd: /home/zgs/.espressif/tools/zephyr/xten sa-esp32-elf/xtensa-esp32-elf/lib/./libc.a(lib_a-locale.o): in function __locale_ctype_ptr’: /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/src/newlib/newlib/libc/locale/locale.c:989: undefined reference to __getreent’ /home/zgs/.espressif/tools/zephyr/xtensa-esp32-elf/bin/…/lib/gcc/xtensa-esp32-elf/8.4.0/…/…/…/…/xtensa-esp32-elf/bin/ld.bfd: /home/zgs/.espressif/tools/zephyr/xten sa-esp32-elf/bin/…/lib/gcc/xtensa-esp32-elf/8.4.0/libgcc.a(unwind-dw2-fde.o): in function __register_frame_info_bases’: /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/src/gcc/libgcc/unwind-dw2-fde.c:100: undefined reference to pthread_mutex_lock’ … /home/zgs/.espressif/tools/zephyr/xtensa-esp32-elf/bin/…/lib/gcc/xtensa-esp32-elf/8.4.0/…/…/…/…/xtensa-esp32-elf/bin/ld.bfd: /builds/idf/crosstool-NG/.build/xtensa -esp32-elf/src/newlib/newlib/libc/stdio/fclose.c:81: undefined reference to pthread_setcancelstate’ /home/zgs/.espressif/tools/zephyr/xtensa-esp32-elf/bin/…/lib/gcc/xtensa-esp32-elf/8.4.0/…/…/…/…/xtensa-esp32-elf/bin/ld.bfd: /builds/idf/crosstool-NG/.build/xtensa -esp32-elf/src/newlib/newlib/libc/stdio/fclose.c:106: undefined reference to pthread_setcancelstate’ /home/zgs/.espressif/tools/zephyr/xtensa-esp32-elf/bin/…/lib/gcc/xtensa-esp32-elf/8.4.0/…/…/…/…/xtensa-esp32-elf/bin/ld.bfd: /builds/idf/crosstool-NG/.build/xtensa -esp32-elf/src/newlib/newlib/libc/stdio/fclose.c:109: undefined reference to __getreent’ collect2: error: ld returned 1 exit status make[2]: *** [zephyr/CMakeFiles/zephyr_prebuilt.dir/build.make:119: zephyr/zephyr_prebuilt.elf] Error 1 make[1]: *** [CMakeFiles/Makefile2:2701: zephyr/CMakeFiles/zephyr_prebuilt.dir/all] Error 2

make: *** [Makefile:91: all] Error 2

I modified ~/github/demo/ai/microtvm/zephyrproject/zephyr/lib/libc/newlib/libc-hooks.c

And applied a patch from Sylvioalves:

git remote add upstream https://github.com/sylvioalves/zephyr.git

git fetch upstream

git cherry-pick 12e9cc5f6b1aa890690e1e8ae074eccf00ed7d38

6. elf too large

I modified the size of dram0_1_seg as above to fix the error:

/home/zgs/.espressif/tools/zephyr/xtensa-esp32-elf/bin/…/lib/gcc/xtensa-esp32-elf/8.4.0/…/…/…/…/xtensa-esp32-elf/bin/ld.bfd: region `dram0_1_seg’ overflowed by 153 824 bytes collect2: error: ld returned 1 exit status make[2]: *** [zephyr/CMakeFiles/zephyr_prebuilt.dir/build.make:119: zephyr/zephyr_prebuilt.elf] Error 1 make[1]: *** [CMakeFiles/Makefile2:2701: zephyr/CMakeFiles/zephyr_prebuilt.dir/all] Error 2

make: *** [Makefile:91: all] Error 2

7. Add flash runner

Add flash_runner for esp32 in tvm/build/microtvm_template_projects/zephyr/microtvm_api_server.py image

After the above tricky steps I can run microtvm_autotune example and got the results in microtvm_autotune.log.txt:

{“input”: [“c -keys=cpu -link-params=0 -model=esp32”, “conv2d_NCHWc.x86”, [[“TENSOR”, [1, 3, 10, 10], “float32”], [“TENSOR”, [6, 3, 5, 5], “float32”], [1, 1], [2, 2, 2, 2], [1, 1], “NCHW”, “NCHW”, “float32”], {}], “config”: {“index”: 27, “code_hash”: null, “entity”: [[“tile_ic”, “sp”, [-1, 3]], [“tile_oc”, “sp”, [-1, 2]], [“tile_ow”, “sp”, [-1, 5]], [“unroll_kw”, “ot”, true]]}, “result”: [[1000000000.0], 7, 100, 1650001086.9535518], “version”: 0.2, “tvm_version”: “0.9.dev0”}

{“input”: [“c -keys=cpu -link-params=0 -model=esp32”, “conv2d_NCHWc.x86”, [[“TENSOR”, [1, 3, 10, 10], “float32”], [“TENSOR”, [6, 3, 5, 5], “float32”], [1, 1], [2, 2, 2, 2], [1, 1], “NCHW”, “NCHW”, “float32”], {}], “config”: {“index”: 87, “code_hash”: null, “entity”: [[“tile_ic”, “sp”, [-1, 3]], [“tile_oc”, “sp”, [-1, 6]], [“tile_ow”, “sp”, [-1, 8]], [“unroll_kw”, “ot”, false]]}, “result”: [[1000000000.0], 7, 100, 1650001189.9725153], “version”: 0.2, “tvm_version”: “0.9.dev0”}

{“input”: [“c -keys=cpu -link-params=0 -model=esp32”, “conv2d_NCHWc.x86”, [[“TENSOR”, [1, 3, 10, 10], “float32”], [“TENSOR”, [6, 3, 5, 5], “float32”], [1, 1], [2, 2, 2, 2], [1, 1], “NCHW”, “NCHW”, “float32”], {}], “config”: {“index”: 31, “code_hash”: null, “entity”: [[“tile_ic”, “sp”, [-1, 3]], [“tile_oc”, “sp”, [-1, 6]], [“tile_ow”, “sp”, [-1, 5]], [“unroll_kw”, “ot”, true]]}, “result”: [[1000000000.0], 7, 100, 1650001297.914291], “version”: 0.2, “tvm_version”: “0.9.dev0”}

{“input”: [“c -keys=cpu -link-params=0 -model=esp32”, “conv2d_NCHWc.x86”, [[“TENSOR”, [1, 3, 10, 10], “float32”], [“TENSOR”, [6, 3, 5, 5], “float32”], [1, 1], [2, 2, 2, 2], [1, 1], “NCHW”, “NCHW”, “float32”], {}], “config”: {“index”: 81, “code_hash”: null, “entity”: [[“tile_ic”, “sp”, [-1, 3]], [“tile_oc”, “sp”, [-1, 1]], [“tile_ow”, “sp”, [-1, 8]], [“unroll_kw”, “ot”, false]]}, “result”: [[1000000000.0], 7, 100, 1650001401.2738233], “version”: 0.2, “tvm_version”: “0.9.dev0”}

{“input”: [“c -keys=cpu -link-params=0 -model=esp32”, “conv2d_NCHWc.x86”, [[“TENSOR”, [1, 3, 10, 10], “float32”], [“TENSOR”, [6, 3, 5, 5], “float32”], [1, 1], [2, 2, 2, 2], [1, 1], “NCHW”, “NCHW”, “float32”], {}], “config”: {“index”: 59, “code_hash”: null, “entity”: [[“tile_ic”, “sp”, [-1, 3]], [“tile_oc”, “sp”, [-1, 2]], [“tile_ow”, “sp”, [-1, 2]], [“unroll_kw”, “ot”, false]]}, “result”: [[1000000000.0], 7, 100, 1650001500.5659993], “version”: 0.2, “tvm_version”: “0.9.dev0”}

{“input”: [“c -keys=cpu -link-params=0 -model=esp32”, “conv2d_NCHWc.x86”, [[“TENSOR”, [1, 3, 10, 10], “float32”], [“TENSOR”, [6, 3, 5, 5], “float32”], [1, 1], [2, 2, 2, 2], [1, 1], “NCHW”, “NCHW”, “float32”], {}], “config”: {“index”: 77, “code_hash”: null, “entity”: [[“tile_ic”, “sp”, [-1, 3]], [“tile_oc”, “sp”, [-1, 3]], [“tile_ow”, “sp”, [-1, 5]], [“unroll_kw”, “ot”, false]]}, “result”: [[1000000000.0], 7, 100, 1650001606.5663857], “version”: 0.2, “tvm_version”: “0.9.dev0”}

{“input”: [“c -keys=cpu -link-params=0 -model=esp32”, “conv2d_NCHWc.x86”, [[“TENSOR”, [1, 3, 10, 10], “float32”], [“TENSOR”, [6, 3, 5, 5], “float32”], [1, 1], [2, 2, 2, 2], [1, 1], “NCHW”, “NCHW”, “float32”], {}], “config”: {“index”: 28, “code_hash”: null, “entity”: [[“tile_ic”, “sp”, [-1, 1]], [“tile_oc”, “sp”, [-1, 3]], [“tile_ow”, “sp”, [-1, 5]], [“unroll_kw”, “ot”, true]]}, “result”: [[1000000000.0], 7, 100, 1650001704.2812748], “version”: 0.2, “tvm_version”: “0.9.dev0”}

{“input”: [“c -keys=cpu -link-params=0 -model=esp32”, “conv2d_NCHWc.x86”, [[“TENSOR”, [1, 3, 10, 10], “float32”], [“TENSOR”, [6, 3, 5, 5], “float32”], [1, 1], [2, 2, 2, 2], [1, 1], “NCHW”, “NCHW”, “float32”], {}], “config”: {“index”: 89, “code_hash”: null, “entity”: [[“tile_ic”, “sp”, [-1, 3]], [“tile_oc”, “sp”, [-1, 1]], [“tile_ow”, “sp”, [-1, 10]], [“unroll_kw”, “ot”, false]]}, “result”: [[1000000000.0], 7, 100, 1650001806.0494645], “version”: 0.2, “tvm_version”: “0.9.dev0”}

{“input”: [“c -keys=cpu -link-params=0 -model=esp32”, “conv2d_NCHWc.x86”, [[“TENSOR”, [1, 3, 10, 10], “float32”], [“TENSOR”, [6, 3, 5, 5], “float32”], [1, 1], [2, 2, 2, 2], [1, 1], “NCHW”, “NCHW”, “float32”], {}], “config”: {“index”: 90, “code_hash”: null, “entity”: [[“tile_ic”, “sp”, [-1, 1]], [“tile_oc”, “sp”, [-1, 2]], [“tile_ow”, “sp”, [-1, 10]], [“unroll_kw”, “ot”, false]]}, “result”: [[1000000000.0], 7, 100, 1650001915.5923896], “version”: 0.2, “tvm_version”: “0.9.dev0”}

But when I try to timing the tuned program, I got stuck:

Leaving…

Hard resetting via RTS pin…

Built target flash

____________serial_port=/dev/ttyUSB0

Traceback (most recent call last):

File “untune_profile.py”, line 97, in

with tvm.micro.Session(project.transport()) as session:

File “/home/zgs/github/tvm/python/tvm/micro/session.py”, line 128, in enter

_rpc_connect(

File “/home/zgs/github/tvm/python/tvm/_ffi/_ctypes/packed_func.py”, line 237, in call

raise get_last_ffi_error()

tvm._ffi.base.TVMError: MicroSessionTimeoutError: session start handshake failed after 5s

Any suggestions to debug with this?

After updating with esp32 toolchain supported newlibc and modify host_driven example code, I can make it run on ESP32.

untune result:

tuned result:

Glad you were able to get this working @oreobird. It was likely that your board had a problem communicating over UART in your original post. In the autotuning logs, “result”: [[1000000000.0] means that there was an error taking that measurement, so the way autotuning performed was consistent with what you were seeing when you tried to run inference. We’re working on improving the way errors are surfaced in autotuning this quarter.