This thread is an embryonic RFC spun out from here.
Problem description
One observation many of us have made is that the µTVM tvm.micro.Compiler interface is a fairly tight integration between TVM and the compiler. The interface is here:
class Compiler(metaclass=abc.ABCMeta):
"""The compiler abstraction used with micro TVM."""
@abc.abstractmethod
def library(self, output, sources, options=None):
"""Build a library from the given source files.
Parameters
----------
output : str
The path to the library that should be created. The containing directory
is guaranteed to be empty and should be the base_dir for the returned
Artifact.
sources : List[str]
A list of paths to source files that should be compiled.
options : Optional[List[str]]
If given, additional command-line flags to pass to the compiler.
Returns
-------
MicroLibrary :
The compiled library, as a MicroLibrary instance.
"""
raise NotImplementedError()
@abc.abstractmethod
def binary(self, output, objects, options=None, link_main=True, main_options=None):
"""Link a binary from the given object and/or source files.
Parameters
----------
output : str
The path to the binary that should be created. The containing directory
is guaranteed to be empty and should be the base_dir for the returned
Artifact.
objects : List[MicroLibrary]
A list of paths to source files or libraries that should be compiled. The final binary
should be statically-linked.
options: Optional[List[str]]
If given, additional command-line flags to pass to the compiler.
link_main: Optional[bool]
True if the standard main entry point for this Compiler should be included in the
binary. False if a main entry point is provided in one of `objects`.
main_options: Optional[List[str]]
If given, additional command-line flags to pass to the compiler when compiling the
main() library. In some cases, the main() may be compiled directly into the final binary
along with `objects` for logistical reasons. In those cases, specifying main_options is
an error and ValueError will be raised.
Returns
-------
MicroBinary :
The compiled binary, as a MicroBinary instance.
"""
raise NotImplementedError()
With mBED (implementation this was originally abstracted from), compilation speed wasn’t so much of an issue, but with Zephyr, this interface has contributed to slow compilation for all uses except autotuning. In particular, some observations:
O1. Given the interface, implementations must set up a clean build environment for each library generated. Depending on the underlying build system (e.g. cmake), this process can take longer than it does to compile the library.
O2. microTVM projects are generally small, especially when compilation is being driven from TVM. Meanwhile, this interface is quite complex and requires TVM to implement abstractions that may or may not work with each project around managing generated code. It may not be worth adding this complexity to TVM.
O3. It’s hard to save generated code to disk right now. If a user wanted to integrate their generated operator code into a project, that’s the next thing they need. Currently, the traditional TVM Module API export_library tries to build a shared object, and we have a function tvm.micro.build_static_runtime which compiles generated code using a TVM-configured compiler. We should fix this, so that it’s possible for TVM to just generate code and get out of the way.
Possible solution
The rest of TVM uses a function fcompile to handle compilation. I think in retrospect, this level of abstraction is about right for the compiler. However, µTVM needs additional things from the build system:
- it needs to flash the built binary onto a target device
- it needs to open a transport (I.e. serial port or socket) to communicate with the device
Finally, there are use cases where only these two things are needed. For example, if you want to debug a problem happening on the microcontroller, you likely want to run with the same firmware image several times.
Therefore, one possible solution is to abstract each of these into a project-centric interface. Here, a project is defined to a standalone set of files that can be compiled into a firmware binary and flashed to a target device. Here is a strawman interface:
class Project(metaclass=abc.abstractclass):
@classmethod
def from_runtime_module(cls, module, workspace) -> Project:
"""When supported, build a new Project in workspace from the generated code in module."""
raise NotImplementedError()
def compile(self) -> MicroBinary:
"""Compile the project and produce a MicroBinary."""
raise NotImplementedError()
def flasher(self, **kw) -> Flasher:
"""Return a Flasher instance that works with this project.
Parameters
----------
kw :
Keyword args that configure flashing or select which board to use.
"""
def flash(self, **kw) -> Transport:
"""Flash the compiled binary in this project's build tree onto the device.
Returns
-------
Transport :
A transport channel that can be used to communicate with the RPC server.
"""
raise NotImplementedError()
Here I’m keeping the MicroBinary so we can attempt to support the following flow:
+-----------------+ +-----------------+ +-------+
| compile machine | --[ TVM RPC ]--> | flasher machine | ---[ USB ]---> | board |
+-----------------+ +-----------------+ +-------+
Next steps
If you have comments or an alternate proposal, please add them here. I’m hoping to work on this in the first few months of 2021.