Note: This pre-RFC was presented at the September TVM Community Meeting (check out the video here). Looking forward to gathering more community feedback!
Summary
This RFC proposes to add product roadmaps to TVM.
Roadmaps should be seen as a way of unifying the planning process of TVM, all the way from ideas to PR merges. The roadmaps discussed in this pre-RFC are intentionally designed to integrate with TVM’s existing planning tools (e.g. Github tracking issues, RFCs), while adding an additional space for sharing and collaboration earlier in the R&D process.
This proposal includes two categories of roadmaps:
- TVM Global Roadmap: This roadmap aims to provide a holistic view of all components within TVM, focusing on features which broadly affect the entire project or add significant functionality.
- TVM Component Roadmaps: These roadmaps aim to provide a more detailed view of a subsection of TVM: including vertical efforts involving TVM’s intermediate representations (such as Relay or TIR) and the horizontal efforts which span across TVM’s full stack (such as Automation or Documentation). Details about each of the components in TVM will be discussed later in this pre-RFC.
Motivations
- Integrate TVM’s technical vision and task tracking mechanisms together
- Provide everyone with visibility into all of TVM’s projects
- Encourage collaborative and open planning in the TVM community
Guide-Level Explanation
This guide-level explanation will describe the basic structure of the roadmap, without focusing heavily on the intended contents.
Tooling
The roadmap utilizes Github Projects as the underlying tooling mechanism. This maximizes reuse of existing content tracked in Github, and is very user-friendly for existing Github users.
This RFC proposes to place the TVM roadmaps directly in the apache/tvm repository, since the roadmap is intended to be a community-focused project, and having the roadmaps directly in TVM would help to maximize visibility to the TVM community.
Sample Roadmap
Above is a sample roadmap for TVM CI and Testing, with annotations to show several important features within the roadmap:
- Background & Motivations Column: This section is located on the left-most part of the roadmap, and is intended to define a few key focus areas of the project and their overall success criteria. In the TVM CI and Testing roadmap, this section contains definitions of Coverage, Ease of Use, Runtime, Stability, and Reporting.
-
Quarterly Planning Columns: Following Background & Motivations, a roadmap has 4 quarterly planning columns (one column per upcoming quarter through the next year). Each Quarterly planning column has these cards:
-
Themes & Goals: The Themes & Goals are listed in the top-most card of each column of the roadmap, and are intended to show how the focus areas of the project are addressed that quarter.
- Themes are chosen from those defined in the Background & Motivations section (such as Stability or Runtime).
- Goals describe how the Roadmap Items listed in each column contribute to the success criteria listed in Background & Motivations.
For example, in Q3 of 2021 (the last 3 months of development in TVM), two of the major focus areas in TVM CI and Testing were Stability and Runtime.
-
Roadmap Items: Each column of development contains roadmap items which contribute to each of the Themes & Goals described above. These items are meant to unify feature development within each project, whether it is coming from an RFC, Github task tracking, or directly from within the roadmap. Items on the roadmap generally have feature ownership at least partially identified at a minimum.
-
- Backlog Column: Items on the roadmap which don’t have an owner or a proposed development timeline fall into the Backlog Items column. This is also the default location where new roadmap issues are placed.
Reference-Level Explanation
Design Considerations
There were several key considerations made while designing this roadmap, centering around the overall themes of quality content, usability, and maintainability. Below are some of these considerations:
- Make it as easy as possible for contributors without domain-level expertise to quickly gain basic context and understanding of a project
-
Encourage consistent and high-quality roadmap content
- Background Information
- There should be a roadmap for all of TVM, and detailed, focused roadmaps for components of TVM
- Each roadmap should contain themes which encapsulate key focus areas of the roadmap
- Each roadmap should contain items which describe either features or milestones
- Each item should have clearly defined motivations, end goals, deliverables, and scope
- Smaller tasks (e.g. fixing flaky tests) may be combined into a larger milestone
- Larger features (e.g. AutoTIR) may be split into smaller milestones
- Schedule & Prioritization
- Priority of any given feature should be determined by consensus, then displayed prominently
- Dependencies and blockers should be easily traceable
- Delivery schedule should be estimated based on prioritization, dependencies, and blockers, and it should be updated as confidence increases
- Ownership
- Subject-matter experts (e.g. feature authors, contributors with subject-matter expertise) should be listed for each feature
- Task-level ownership should be listed for each feature
- Tasks with help needed should be discoverable to new users
- Background Information
-
Leverage existing task tracking mechanisms
- Easier maintenance for everyday contributors
- Show updated progress towards feature completion
-
Encourage sharing and collaboration in the early stages of planning
- Roadmap describes what changes are desired
- RFC describes how changes will be implemented
Proposed Roadmap Structure
Below is the initial set of roadmaps proposed in this RFC.
| Roadmap Name | Scope |
|---|---|
| TVM Global Roadmap | All of TVM’s efforts |
| TVM Automation Roadmap | Automated performance improvements across the entire TVM stack |
| TVM Community Roadmap | Community efforts across all of TVM, including Developer Tooling, Documentation, Events, Videos, and Tutorials |
| TVM Continuous Integration & Testing Roadmap | CI and testing across all of TVM |
| TVM Core Compiler Roadmap | Major compiler features and/or refactors with impacts across key APIs and/or multiple IRs in the TVM stack |
| TVM Frontend & Interface Roadmap | Major user-facing pieces of TVM, such as importers, TVMC, and user facing APIs |
| TVM Graph Computation & High-Level Optimizations Roadmap | All things Relay related |
| TVM Scheduler & Low-Level Optimizations Roadmap | All things TIR related |
| TVM Hardware Roadmap | All things hardware backend related |
| microTVM roadmap | All things microTVM related |
Challenges
There are a few challenges to having publicly available roadmaps, but overall, we believe that the benefits of having roadmaps for TVM vastly outweigh the negatives.
- Changes to the roadmap: Items on the roadmap are subject to change as plans and community consensus changes. It is inevitable that projects get delayed or rescoped with time, and this will impact the roadmaps. However, the true value of roadmapping is to empower contributors to share their ideas, changes included, with the community.
- Contents of the roadmap: TVM has a broad and diverse contributor base who may take different approaches to authoring and reading the contents of the roadmap. In order to ensure uniformity and high-quality contents within the roadmap, community roadmap guidelines will be created and upstreamed.
-
Organization of the roadmap: Since the proposed roadmaps would live directly in
apache/tvmvia Github Projects, roadmap items would be created directly inapache/tvmas Github issues. Currently, Github issue usage is very limited inapache/tvmin order to ensure easier triage and timely closure of issues. Having the TVM roadmaps live inapache/tvmwould increase the overall count of Github issues in the repository, but the roadmap items could be labeled with atvm:roadmaptag so that they can be properly triaged and filtered.
Rationale & Alternatives
There are several alternatives to this roadmap proposal; however, each of the alternatives has drawbacks of its own.
- Do nothing: Don’t create a publicly viewable roadmap. However, the community would miss out on the numerous benefits of having a roadmap.
-
Create a separate repository for roadmap: Having a separate roadmap repository (e.g.
apache/tvm-roadmap) might reduce the number of roadmap issues filed directly toapache/tvm, but it would increase the complexity and fragmentation of the roadmap significantly, since this roadmap repository would have to link to RFC tracking issues inapache/tvmanyways. - Use a separate tool for the roadmap: Using a separate tool for the roadmap may provide more functionality than Github projects. However, this would require much more effort to create and maintain.
Prior Art
This RFC is inspired by the previous work done to improve the tracking mechanisms of TVM. This includes but is not limited to the existing RFC process, release process, and technical visions of TVM.
It is also inspired by the public Github roadmap, which is an excellent example of usage for Github Projects.
Upstreaming Timeline
This is a prospective upstreaming timeline for the TVM roadmap.
| Milestone Name | Deliverables | Schedule |
|---|---|---|
| TVM Roadmap M1A: Taking the Plunge |
|
Late September |
| TVM Roadmap M1B: Building Momentum |
|
Mid October |
| TVM Roadmap M1C: Ready for Prime Time |
|
Early November |
Unresolved Questions
Questions about the design and contents of the TVM Roadmap will be progressively resolved through the upstreaming timeline shown above. Any unresolved questions at the end of TVM Roadmap Milestone 1 will be created as roadmap items in the TVM Community Roadmap 
Future Possibilities
- Create Github issue templates for roadmap items (similar to the existing Github issue templates for docs, CI, etc).
- Use the new Github issues/projects (closed beta) to improve roadmap functionality.
Special Thanks
Thanks so much for copyediting, @areusch and @electriclilies! 
Looking forward to releasing documentation outlining this in M1B.