Standalone execution is possible after all?

sho · November 25, 2021, 10:10am

Thank you for visiting my question.

I’m now trying to run inference on bare-metal devices. More specifically, I’d like to use micro tvm(or the outputs from micro tvm) on MCUs.

However, I went through some tutorials or notebooks created by developers:

github.com

areusch/microtvm-blogpost-eval/blob/master/tutorial/standalone_utvm.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# MicroTVM Host-Driven Example\n",
    "\n",
    "**Author**: `Andrew Reusch <areusch@octoml.ai>`_\n",
    "\n"

This file has been truncated. show original

and found that MCUs must be connected to the host machine anyway.

Is it possible for me to download some binary into MCUs, and run inference without connecting to the host PC just as I can do with TFL micro?

I’m not sure when MCUs have to communicate with the host PC to run inference, and when standalone MCUs can run inference without being connected to the host PC.

Could anyone help me understand the mechanism?

grant-arm · November 25, 2021, 12:35pm

Hi @sho ,

If you’re looking for a demo of a standalone application running on an MCU, you could take a look at https://github.com/apache/tvm/tree/main/apps/microtvm/ethosu . Although this demo is an example of how to use TVM to run a model and offload operators to the microNPU, it should provide a good starting point for running standalone on bare metal.

sho · November 25, 2021, 1:05pm

@grant-arm Thanks a lot for the link. I’ll have a look at the repo and see if standalone execution is possible for any MCUs.