Understanding the relationship between microTVM, autoTVM, RPCRunner, rpc_tracker and MicroTVMRpcServerLoop

[fPecc]

Hi all!

I have currently been trying to understand the relationship between this components, in order to integrate a custom accelerator + board into TVM.

As a reference, this is what I would like to try:

• AutoTVM running on the host computer
• Multiple boards connected to the same host computer running the AutoTVM schedules.

Now, each of this boards only runs baremetal applications, so I understand that I should have a simple application running in each of them, using the MicroTVMRPCServer (similar to the host driven Zephyr example).

Now in my host computer, I have the ProjectAPI of my specific configuration, already coded to communicate through UART with the board (also similar to the Zephyr example).

What I am failing to understand is how the tvm.autotvm.RPCRunner and the rpc_tracker integrate with this. All examples I have seen use the RPCRunner to point to an already started rpc_tracker, which should have boards (or hardware) already registered in it.

So the questions are:

1. Are there any examples running AutoTVM on baremetal devices using microTVM? (there is one for microTVM but it uses the host as target).
2. Should I write my own runner in order to communicate with my board through the UART? Or should the RPCRunner be able to manage this?

Thanks all!

[areusch]

@fPecc thanks for your questions! Here are some answers.

@fPecc

• AutoTVM running on the host computer
• Multiple boards connected to the same host computer running the AutoTVM schedules.

We’ve demonstrated the first one, but not the second one yet. It should be possible through Project Options to identify the board serial number you’d like to use for the runner.

@fPecc

Now, each of this boards only runs baremetal applications, so I understand that I should have a simple application running in each of them, using the MicroTVMRPCServer (similar to the host driven Zephyr example).

That’s correct.

@fPecc

What I am failing to understand is how the tvm.autotvm.RPCRunner and the rpc_tracker integrate with this. All examples I have seen use the RPCRunner to point to an already started rpc_tracker, which should have boards (or hardware) already registered in it.

Great question. At present microTVM uses AutoTvmModuleLoader to override the process of connecting to a proxy TVM RPC server (running on a machine connected to the physical hardware) and building/flashing firmware (this is done at the time of module loading rather than in the AutoTVM Builder because most firmware build systems do not like their build artifacts to be moved).

You can create an RPC tracker and connect several instances of TVM RPC server to that tracker. However, the project options are not currently customized per-RPC-server, so we still have a task to be able to override the serial number via project options to support multiple boards.

@fPecc

1. Are there any examples running AutoTVM on baremetal devices using microTVM? (there is one for microTVM but it uses the host as target).

You should be able to get it working by substituting a bare metal target for the host one. There is an example there in comments in one of the code snippets (Ctrl+F for Compiling for physical hardware).

@fPecc

Should I write my own runner in order to communicate with my board through the UART? Or should the RPCRunner be able to manage this?

AutoTvmModuleLoader should abstract this away so that you can use RPCRunner. You do need to implement this communication in your microTVM Project API server.

Hope this helps!

[fPecc]

@areusch Thanks for the answer!

So, the AutoTvmModuleLoader eliminates the need to run the rpc_server on the target board, correctly? This would also be impossible because its a baremetal application the one running on the board, so thats ok.

Just to confirm: the rpc_tracker doesn’t need to be running in order to tune the tuner.

[areusch]

sorry for the delay here–no, you don’t need an RPC tracker. in the future, when we support distributing tuning jobs to more than 1 board, a tracker may be needed functionally, but i suspect we would hide that under the covers.