I am a PhD student and have been working with TVM for over a year. My main field of research is TinyML and RISC-V based microcontrollers. Thus, I work on supporting bare metal 32-bit RISC-V targets (Actual hardware and simulators) using the MicroTVM Platform.
Over the past years, there has been a lot of interest in supporting RISC-V hardware in the TVM Compiler Suite. While some attempts have been made in the past, nothing has ended up upstreamed in the TVM repository. As several teams/individuals seem to be working in the same direction at the moment, it would make sense to align these efforts to reduce the risk of duplicate work and agree on some relevant design decisions beforehand.
While RISC-V is not yet officially supported by TVM, it can already be used using the default LLVM-based flow or the MicroTVM APIs (via the RISC-V QEMU running on the Zephyr Platform). However, the generated code (falling back to default or ARM schedules) is not yet optimized for the RISC-V ISA. Thus performance will not be optimal.
In my opinion, the following baseline tasks:
- Integrate support for RISC-V ISA simulation (ISS) in TVMs CI scripts and unit tests.
- Decide how to detect the available RISC-V ISA extensions. Align this with the work done for ARM Cortex-M MCUs?
- Add RISC-V specific schedules (default, packed (sub-word SIMD), and vector (super-word SIMD) in TVM: either starting with generating schedules or based on the current
arm_cpuschedules. (See tracking issue: https://github.com/apache/tvm/issues/10141)
- We also eventually need type legalizations, alter_op_layout, and the operator strategies
- Generate AutoTVM logs for relevant models on RISC-V hardware and add them to the top hub (https://github.com/tlc-pack/tophub) repository. This should help to get better default performance for some use cases.
As it is quite hard to follow all the attempts to integrate RISC-V related features into TVM I‘ve tried to compile a list of efforts in that direction. Please let me know if I missed something relevant.
[RFC][Backend] RFC-CSI-NN2-Integration (@alter-xp)
- Integration of CMSIS-NN like vendor library by Alibaba’s T-Head bringing support for the RISC-V C906 (RVV0.7.1) Linux-capable hardware (e.g. Alwinner D1) and future RVV1.0 chips
- Following the BYOC approach
- Custom QUEMU build required for CI
- Discussion: How to determine the available extensions:
-mcpu & -mattr(https://github.com/apache/tvm-rfcs/pull/75#discussion_r892910809)
- Pre-RFC: [pre-RFC][ BYOC] RISC-V CSI-NN2 Compute Library integration
- RFC PR: https://github.com/apache/tvm-rfcs/pull/75
- RFC: https://github.com/apache/tvm-rfcs/blob/main/rfcs/0075_RISC-V_CSI-NN2_Intergration.md
- Tracking Issue: https://github.com/apache/tvm/issues/11506
[RFC] Adding initial SVE implementation (@MeeraN7)
- Vectorization support for ARMs scalable vector extension has not landed yet in TVM.- However, there are efforts in a good direction. It is likely that some of these works will be helpful for RISC-V, as well as the RVV extension is quite similar to the SVE.
- RFC PR: https://github.com/apache/tvm-rfcs/pull/18
- Related PR: https://github.com/apache/tvm/pull/8655 (outdated)
[RFC] Enable TVM QNN on RISC-V with Subword SIMD Computation (@yrchen)
- This 2 year-old proposal by the NTHU intended to add SIMD support for RISC-V targets with the P-extension.
- It covers mainly optimized schedules and tensorization (vectorization) for the dot-product intrinsic
- A custom RISC-V (Spike) runtime was proposed as well. Unfortunately, nothing of the work got upstreamed.
- Pre-RFC: [RFC] Enable TVM QNN on RISC-V with Subword SIMD Computation
[μTVM] Deployment on GAP8 RISC-V platform (@JosseVanDelm)
Our reimplementation of ARMs CMSIS-NN kernel library for RISC-V targets supporting the latest spec of the packed (RVP) and vector (RVV) extensions and a CMSIS-NN compatible interface.
Discuss Post: Integration of muRISCV-NN kernel library in TVM
In an older version of microTVM, a device called
riscv_spikewas introduced to run simulations using the
riscv-isa-sim. Unfortunately, this was dropped over one year ago. It would be desirable to revive this feature sometime in the future (if possible)
In addition, some TVM-related talks have been held at some RISC-V events. Unfortunately, I could not find any implementation code online.
- Enabling TVM on RISC-V Architectures with SIMD Instructions: Enabling TVM on RISC-V Architectures with SIMD Instructions - YouTube
- Support TVM QNN Flow on RISC-V with SIMD Computation - Yi-Ru Chen & Jenq Kuen Lee: Support TVM QNN Flow on RISC-V with SIMD Computation - Yi-Ru Chen & Jenq Kuen Lee - YouTube
- Lightning Talk: Performance of TVM AutoScheduler for Andes Vector Processor - I-Wei Wu: Lightning Talk: Performance of TVM AutoScheduler for Andes Vector Processor - I-Wei Wu - YouTube