BYOC Partitioning Support on TVMC
In this RFC, we are proposing an implementation that allow users to reach BYOC targets using TVMC. This is done by applying the expected sequence of passes, in the order requested by the user via command line.
In the current implementation, users can’t target BYOC backends directly using TVM Target API, needing to resort to custom passes execution using Python custom scripts.
This RFC proposes a mechanism to expose BYOC targets to end-users via TVMC, as a way to offer that feature via command line. It does so by linking the target definitions passed by
--target, with the series of passes and partitioning that currently needs to be done manually.
In the specific case of BYOC, TVMC needs to invoke manual partitioning and a set of passes that will adjust the graph for our expected BYOC.
One or more BYOC implementations are expected to be targeted - together or separate - in any (reasonable) combination the user might want.
To accomplish this, the present RFC expects to extend the existing
--target option syntax, to accommodate the target(s), in a way to support BYOC, as well as keeping the target option as a consistent way to define a target.
tvmc compile --target="ethos-n77, llvm -mtriple=aarch64-linux-gnu -mattr=+neon" ...
The sets of passes to be executed for a given BYOC are implemented by a class, that will link the command line option with that BYOC target. This class also will be able to translate the options given via command line, to be translated to a dictionary passed to the
tvm.transform.PassContext, at compilation time.
Together with this RFC, I’ll soon submit a PR https://github.com/apache/tvm/pull/7304 that implements what is here. We are currently using this approach successfully to test the codegen for Arm Ethos-N NPU. This will also be included the PR.