[pre-RFC] Roadmap RFCs

[denise]

Co-authored-by: @areusch

This Roadmap-RFC Process is merely a proposal. We are submitting this pre-RFC to collect feedback from the community. We encourage everyone to reply and help us improve this proposal.

Background

The Roadmap-RFC Process Proposal is a follow-on to [pre-RFC] TVM Roadmap posted in the Discuss forum earlier this year. In discussion on this pre-RFC, it was noted that there wasn’t a way for community members to directly modify the Roadmaps. While the intent of the original RFC was merely to reflect the work being done by the community, the community needs a way to engage with the roadmaps directly if they are to be used for project planning.

The goals of the Roadmap-RFC Process Proposal are:

• To follow up on comments/questions raised on [pre-RFC] TVM Roadmap about interacting and upstreaming the initial roadmap items.
• Enable individuals and organizations that contribute to TVM to influence the vision of TVM via roadmaps
• Encourage individuals and organizations that contribute to TVM to share the details of their longer-term efforts
• Ensure a openly-governed, equitable process towards creating new roadmaps for TVM

Overview

This RFC describes a process for the community to directly propose changes to TVM Roadmaps, including:

• Creating new roadmaps and updating existing roadmaps
  • Template of a roadmap’s contextual information
  • Establishing the scope and themes behind a roadmap
• Adding items to roadmaps

Components of a Roadmap

The key components of a roadmap are discussed in [pre-RFC] TVM Roadmap - below is a quick recap:

• Background & Motivations: This section is intended to define a few key focus areas of the project and their overall success criteria.
• Quarterly Planning Columns: there are 4 quarterly planning columns defined on a roadmap. Each column contains:
  • Themes & Goals: intended to show how the focus areas of the project are addressed that quarter
  • Roadmap Items: Track the actual tasks being accomplished this quarter. Roadmap Items are GitHub Issues which either track work (such as pre-RFCs, RFCs, or implementations of those) or track efforts to fix bugs.
• Backlog Column: Items on the roadmap which don’t have an owner or a proposed development timeline fall into the Backlog Items column. This is also the default location where new roadmap issues are placed.

Process Definitions

This section proposes the Roadmap-RFC process, which aims to encapsulate the processes discussed in the Overview section. The Roadmap-RFC is used to create, modify, or delete roadmaps. Details on the process are listed below.

Template of a Roadmap’s Contextual Information

The following fields provide some contextual information for each roadmap.

• Roadmap Name: Name of the roadmap
• Roadmap Maintainers: These community members are primarily responsible for adding, modifying, and removing roadmap items. Any community member can be listed as a maintainer of a roadmap, regardless of contributor status within the Apache organization.
  • Note: If a Roadmap-RFC’s proposed maintainer does not have TVM Committer or PMC status within the Apache organization, they will receive an invitation for a Triage role in Apache TVM as soon as their Roadmap-RFC is accepted. This will allow them to maintain and triage roadmaps while maintaining the Apache community contribution process.
• Roadmap Summary/Description: A brief description of the roadmap

Establishing Scope and Themes for a new Roadmap

For each roadmap, a set of scope and themes should also be defined, in order for TVM’s community members to quickly learn about a roadmap’s key focus areas.

Establishing Scope

• How are the tasks tracked in this roadmap grouped together? How can we think about this grouping distinct from those made in other roadmaps? It’s okay for there to be overlap with other roadmaps, but the scope defined here should motivate a separate roadmap.
• Is the proposed roadmap intended to represent a perpetually ongoing set of efforts, or is there an end goal which will close/finalize the roadmap?
• Does the proposed roadmap have any scope overlaps with any existing roadmaps? If so, please list them.

Establishing Themes

• List 4-6 proposed “themes” of the roadmap, intended to convey the purpose of the roadmap and the types of tasks that should be added.
  • Some examples of themes are programmability, portability, and performance.
  • Themes are intended to group items within a roadmap. This helps us to understand the scope of the roadmap.
• For each theme, include a set of definitions specific to the proposed roadmap.
  • What does this theme mean in the context of this roadmap?
  • Are there multiple definitions for this theme? For example, performance could be interpreted as tuning times, runtime latency, or a number of other definitions.
• For each theme, include a set of success criteria specific to the proposed roadmap.
  • What types of metrics would be relevant to this roadmap?

Adding Items to a Roadmap

A roadmap’s maintainers are the primary folks responsible for adding, modifying, and removing items for a roadmap. This isn’t a hard and fast rule—for example, it may be expedient for other community members to triage new roadmap items into a roadmap’s backlog. However, the maintainers should be considered the “owners” of a roadmap, and generally no rigid process is defined around modifying the items in a roadmap.

Roadmaps are defined using GitHub Projects and can include GitHub Issues, Pull Requests, and simple note cards. Maintainers should strive to place mainly GitHub Issues in Roadmaps to make it possible for the community to learn more about ongoing work and reduce triage burden.

Each item on a roadmap is intended to track one of these community processes:

• pre-RFCs serve as a way to begin discussions on planned work at the earliest stage of maturity. pre-RFCs are typically posted in the TVM Discuss forums in order to solicit TVM Community feedback. For an example of a pre-RFC, see the screenshot of Andrew R’s proposal to Convert RST Docs to Markdown below.

  pre-RFCs can be tracked on a Roadmap by preemptively creating a GitHub Task-tracking Issue in tvm-rfcs.

  

• RFCs serve as a way to share feature proposals with the TVM Community. The tvm-rfcs repo is used for the creation and acceptance of RFCs. For some examples of RFCs, see the screenshot of open pull requests in tvm-rfcs below.

  Open RFCs can be directly linked into any roadmap. Once an RFC is accepted, please use the Github Task-Tracking process to track RFC Execution.

  

• Github Task-Tracking Issues are used in tvm to share the progress of accepted RFCs over time. For an example of a Github Task-Tracking Issue, see the screenshot of Andrew L’s RFC to Add Mixed-Precision Support to TVM below.

  These task-tracking issues can be directly linked into any roadmap.

  

• Bugfixes are actionable GitHub Issues not necessarily connected to a Task-Tracking Issue. Generally, they require only 1 PR and the work is clearly specified in the issue. For an example of a bugfix Github Issue, see the screenshot of a flaky CI test report below.

  You can directly link bugfixes in a roadmap by adding their corresponding GitHub Issues as a card. Generally, it’s the maintainer’s responsibility to combine related bugfixes in order to keep the roadmap concise. For example, if you have 10 flaky tests in CI, it might make sense to have a global tracking issue for flaky CI tests which points to each individual bug.

  

Questions for Discussion

1. Is this process worth engaging with? Do you feel it will overall help the community get a birds-eye view of the ongoing developments in TVM?
2. What happens if a roadmap becomes unmaintained?
3. How should we connect the dots between a roadmap and the TVM release process?