Issue Triage Workflow

Issue Triage

  • Feature Name: Update Issue Triage Workflow
  • Co-authored with @areusch, @denise


This RFC proposes expanding functionality to the existing “New Issue” templates in order to allow for more a precise logging and tracking system for GitHub Issues. The updated templates will be leveraging a new triage-bot to automatically add relevant labels and cc the appropriate person(s). This RFC also includes establishing more current labels while discontinuing the use outdated labels (see bottom of the RFC for the list).


Currently, it is sometimes difficult to find and organize TVM’s GitHub Issues due to the lack of specificity. More specifically, our current means of classifying GitHub Issues is the label attribute, and many of the labels help only to identify the kind of issue (bug, feature, docs, etc). Most of our labels only vaguely correspond to logical components of the TVM codebase (for instance: type: backend — which backend?) and no longer aligns with the latest dev efforts. It is currently difficult, for example as a BYOC owner, to identify issues that the community believes should be resolved through changes to a BYOC implementation in TVM.

As the codebase grows, there is increasingly a need to be able to organize Issues according to the logical component of the codebase affected. Doing this allows folks with specific expertise and ownership to focus on a smaller set of issues. It also allows us as a community to more concretely establish issue triage as a role, increasing opportunity for advancement through non-code contribution.

GitHub Issue Templates can help with this, but in TVM these are primarily organized around the issue type (bug, feature, docs, etc) rather than sorting an issue by component.

This RFC hopes to more naturally organize TVM’s GitHub Issues by introducing new labels and leveraging a new triage-bot which can parse the issue title and body to assign labels and cc the appropriate contributors.

The overall goal of this RFC is to simplify and strengthen the issue triage workflow, in turn allowing the TVM community to more easily navigate and contribute to the project.

Guide-level explanation

Label Reorganization

Firstly, this RFC proposes to deprecate a number of the labels we currently use to identify codebase components and add a new set. The new set is much larger than the previous set; however, we believe the new set corresponds to the logical structure of TVM with more granularity. As discussed above, we think it’s important that folks volunteering to maintain a component have an easy way to identify the set of tasks/issues that need their attention.

To that end, we suggest the a new set of labels. We also suggest to modify []( to reflect each Reviewer and Committer’s expertise in terms of these labels. As future work, the Auto-CC subscription tracker can then automatically better tag those with relevant expertise on relevant issues.

GitHub Issue Templates

This RFC plans to phase out Update CI Docker Image going forward as there is no longer a strong need for it. The rest of the templates will stay.

The figure below shows the templates TVM uses now. We suggest to modify all remaining templates to assign a needs-triage label by default. This new label will alert either the new triage-bot or TVM contributors to sort the issue.

When a new issue is created with the needs-triage label, the triage-bot will parse the issue’s title or text body (see below for a discussion of options) for label tags. If one or more is found, triage-bot will remove needs-triage and add the found labels to the issue.

This RFC proposes to modify the :bug: Bug Report and :wrench: Feature Tracking templates to add instructions encouraging users to self-select which component an issue should belong to. These templates will link to a Markdown doc (checked into TVM) that lists all the label tags and includes a small description for the scope of each one. This will help the users find the appropriate label tags to use.

Example User Workflow

In these templates, the triage-bot will parse the issue’s text body for tags in bullet form.

For example, when creating an issue concerning a bug in autotvm , the process would be -

  1. The user would select the Bug Report template and will see a section shown in body to list the tags they would like to use. It will include a link to the document mentioned above which lists all the label tags.

  2. The user would then add the relevant tags. In this case, they would add autotvm.

    In the case that the issue creator is unsure of which label tags to use, they can simply leave the label tag section blank. This template will add the needs-triage label by default which can alert other TVM contributors who can assist in properly sorting the issue.

  3. After, filling out the quick summary of the issue, they would post it. It will look like shown in this example here and will trigger the triage-bot to parse the issue body and add the appropriate label.

  4. Then, the triage-bot removes the needs-triage label and adds the appropriate label. In this example, it would add autotvm.

If triage-bot is unable to find any labels, it will leave the issue alone. The needs-triage label will continue to indicate issues which need manual triage. We hope by establishing this category that it encourages more community members to work together to triage issues by consolidating those that need it into a single location.


This will improve quality of the issue tracking and make it easier to drill down on specific pain points for different components. Users will be able to quickly sort the issues by their areas of interests and needs.

Issues can have multiple labels tagged, although we ask community members to exercise their judgement and file issues in as few components as the situation necessitates.

Contributors can propose a new label by posting on the Discuss forum. Requests should include adequate scoping to justify the label, as was done in this doc. Labels can similarly be retired via a proposal on Discuss.

Migration Work

After adding the new labels, we will take a couple of migration steps to ensure parity with other codebase descriptions:

  1. Re-sort open issues into the new labels.
  2. Modify []( to identify committer areas of knowledge by reference to the labels.


While the changes this RFC is proposing can be beneficial, they can be confusing if used inexactly.

Possible pitfalls include improper tagging in issues leading to incorrect labels or lack of labels all together. This can create a time sink for other contributors who will need to correct the titles and/or labels.

Another risk is creating bloated issues with far too many labels. This RFC relies on the community’s best judgement when opening issues with the hope that an issue will not have excessive labels.

The net amount of labels this RFC proposes to deprecate and add will result in more labels than before. This can cause some confusion among the community. One way to mitigate is to publish a list of the new labels and a quick description for each one. The document can be referenced by the community.

Rationale and alternatives

As the TVM community grows with diversifying components and interests, it is important to build and maintain a simple but robust issue tracking workflow.

While the current process works, it can sometimes be tedious to manually assign labels and cc appropriate people. This can cause issues that slip through the triage process and fail to be assigned to the right person(s).

This RFC hopes to scale up the issue tracking workflow, introduce more automation and reduce time spent on manual triaging.


Label Schema

There are a few alternatives to the proposed schema that might shorten the label schema and make for easier typing:

  1. Less aggressive grouping. Specifically: remove tune: , dev:, vert: labels
  2. Fewer subcomponents. Specifically: group relay:ir, relay:op, relay:parser, relay:printer into relay:core. Also group tir:analysis and tir:arith into tir:analysis, and tir:ir, tir:op, tir:printer, tir:parser into tir:core.

Issue Filing Flow

  1. Title - Instead of including the label tags in the issue body, the user can include them in the issue title. The triage-bot will then parse the title as opposed to the issue body. This plan will still include a document that lists all the label tags with short descriptions.

    Pros: Can be more intuitive to some users. It will also lead to a shorter issue template since the “Triage” section won’t be needed anymore.

    Cons: Adding multiple label tags will result in a long issue title making it harder to read.

  2. Dropdown - The dropdown approach will still only introduce one template that will include drop downs sections for each high-level component (see example below). The user will remove the tags that are irrelevant leaving behind only ones that are needed.

    Pros: Remove the need to refer to the documentation that lists all the label tags.

    Cons: The issue template will be long due to all the label tags being listed.

  3. Component Specific Issue Templates - This approach requires that each high level component such as “frontend” or “codegen” (identified in the labels below with <component>: prefixes) have its own issue template. Each of these high level component templates will include a list of its sub-components in the issue body that the user can prune to only post sub-components that are relevant to their post.

    Pros: Remove the need for another page with all the label tags and their descriptions. Can also provide some organization structure from the start of the triage process.

    Cons: Limits opportunities to open issues that include multiple high-level components. Results in a long list of issue templates and can look bloated to some users.

Future possibilities

The goal is to establish a solid issue triage workflow that will be flexible to the community’s needs as well. As TVM continues to grow, the labels used will also evolve.

As a possible next step, it would be beneficial to gather a more comprehensive roster of responsible person(s) for each component and automatically assign them to issues. A roster exists now that would need to be updated and formalized to take this effort further.

It would also be helpful to change the auto-subscription cc tracker to use this list of labels as the list of topics that can be subscribed.

New Labels

This section sketches most of the labels we anticipate creating. Upon initial triage (or in response to community feedback), we may create more.

  • byoc:<*codegen_backend*> for each BYOC codegen backend in the compiler. Possible labels shown below:

    Label Scope
    byoc:ethosu python/tvm/relay/op/contrib/
    byoc:dnnl python/tvm/relay/op/contrib/
    byoc:cudnn python/tvm/contrib/
    byoc:cmsis_nn src/relay/backend/contrib/cmsis_nn
    byoc:codegen_c src/relay/backend/contrib/codegen_c
    byoc:codegen_json src/relay/backend/contrib/codegen_json
    byoc:cublas src/relay/backend/contrib/cublas
    byoc:cudnn src/relay/backend/contrib/cudnn
    byoc:cutlass src/relay/backend/contrib/cutlass
    byoc:dnnl src/relay/backend/contrib/dnnl
    byoc:ethosn src/relay/backend/contrib/ethosn
    byoc:ethosu src/relay/backend/contrib/ethosu
    byoc:example_target_hooks src/relay/backend/contrib/example_target_hooks
    byoc:libtorch src/relay/backend/contrib/libtorch
    byoc:tensorrt src/relay/backend/contrib/tensorrt
    byoc:uma src/relay/backend/contrib/uma
    byoc:verilator src/relay/backend/contrib/verilator
    byoc:vitis_ai src/relay/backend/contrib/vitis_ai
  • backend:*<builds>* for each possible backend in TVM (covers codegen and emitted runtime::Module). Possible labels shown below:

    Label Scope
    backend: aocl src/target/source/
    backend: aocl_sw_emu src/target/source/
    backend: c src/target/source/
    backend: cuda src/target/opt/
    backend: hybrid
    backend: llvm src/target/llvm/
    backend: metal src/target/source/
    backend: nvptx src/target/llvm/
    backend: opencl src/target/source/
    backend: rocm src/target/llvm/
    backend: sdaccel src/target/source/
    backend: stackvm src/target/stackvm/
    backend: vulkan src/target/spirv/
    backend: webgpu src/target/spirv/
  • core:<part> for each core piece of TVM

    Label Scope
    core:ffi python/tvm/_ffi
    core:object python/tvm/runtime/
    core:rpc src/runtime/rpc
  • dev:<*category*> for each category of developer infrastructure in TVM. Possible labels shown below:

    Label Scope
    dev:ci issues regarding TVM CI, including scripts living in the TVM repo and configurations living in other public repositories
    dev:test-infra issues regarding test helpers within TVM, e.g. helpers within unit or integration test code
    dev:tools issues regarding TVM developer-facing tools, e.g. code within tools after the dev tools reorg lands
  • docs for any issues regarding TVM’s documentation.

  • flow:<*codegen_path*> for each graph-level codegen flow in the compiler. Possible labels shown below:

    Label Scope
    flow:aot src/relay/backend/
    flow:graph src/relay/backend/
    flow:vm src/relay/backend/vm/*
  • flow:<*lowering_path*> for each lowering flow in the compiler. Possible labels shown below:

    Label Scope
    flow:relay The overall lowering flow for, including BYOC core. Excludes the TIR portion of the flow in flow:tir. python/tvm/relay/, src/relay/backend not covered by other flow execution paths
    flow:tir The overall lowering flow for schedules and TIR ( python/tvm/driver, src/driver.
  • frontend:<*framework*> for each importer in TVM. Possible labels shown below:

    Label Scope
    frontend:caffe python/tvm/relay/frontend/caffe*
    frontend:coreml python/tvm/relay/frontend/
    frontend:darknet python/tvm/relay/frontend/
    frontend:keras python/tvm/relay/frontend/
    frontend:mxnet python/tvm/relay/frontend/mxnet*
    frontend:nnvm python/tvm/relay/frontend/
    frontend:oneflow python/tvm/relay/frontend/
    frontend:onnx python/tvm/relay/frontend/
    frontend:pytorch python/tvm/relay/frontend/torch
    frontend:tensorflow python/tvm/relay/frontend/tensorflow*
    frontend:tflite python/tvm/relay/frontend/tflite*
  • relay:<*component*> for each component of relay. Possible labels shown below:

    Label Scope
    relay:analysis src/relay/analysis
    relay:collage src/relay/collage
    relay:ir src/relay/ir
    relay:op src/relay/op
    relay:parser Relay parser: src/parser
    relay:printer Relay printer: src/printer/
    relay:qnn src/relay/qnn
  • rpc for the RPC protocol and its common implementation (across C and c++ runtimes).

  • runtime:<*library*> for each runtime library. Possible labels shown below:

    Label Scope
    runtime:c C runtime: src/runtime/crt, excluding RPC.
    runtime:c++ C++ runtime: src/runtime, excluding crt and RPC
  • topi for templated operators in python/tvm/topi.

  • tir:<*component*> for each component of tir. Possible labels shown below:

    Label Scope
    tir any TIR core issues which don’t fit into any of the following categories (include/tvm/tir, src/tir)
    tir:analysis src/tir/analysis excluding src/tir/analysis/contrib
    tir:arith src/arith
    tir:ir src/tir/ir, and wrappers in python/tvm/tir
    tir:op src/tir/op, and wrappers in python/tvm/tir
    tir:printer TIR printer:python/tvm/script, src/printer/, src/printer/
    tir:parser TIR parser: python/tvm/script
    tir:schedule S-TIR and utilities: src/tir/schedule and python/tvm/tir/schedule
    tir:transform TIR transforms: src/tir/transforms and python/tvm/tir/transforms
  • tune:<*tuner*> for each tuning technology. Possible labels shown below:

    Label Scope
    tune:auto_scheduler src/auto_scheduler, python/tvm/auto_scheduler
    tune:autotvm src/autotvm, python/tvm/autotvm
    tune:meta_schedule src/meta_schedule, python/tvm/meta_schedule
  • vert:<*component*> for each vertical component of TVM. Possible labels shown below:

    Label Scope
    vert:android issues affecting Android vertical
    vert:hexagon issues affecting Hexagon vertical
    vert:ios issues affecting iOS deployment vertical
    vert:micro MicroTVM: src/runtime/micro, src/runtime/crt, apps/microtvm
    (additional added on request or initial triage)

Proposed Labels to Deprecate

High-level component : specific section

  1. Backlog
  2. Core complier reorg
  3. microtvm:arch
  4. priority:high
  5. priority:low
  6. priority:medium
  7. status: api design review
  8. inactive
  9. status: need experimental markup
  10. status: review in progress
  11. status: RFC
  12. status: stale
  13. status: superseded
  14. status: vote
  15. type: backend
  16. type: installation
  17. type: perf-regression
  18. type: question
  19. type: topi (move to topi label)

We discussed this in the TVM Community Meeting this morning. Here are notes (thanks @csullivan for taking these):

  • Main goal: Automate assigning labels to issues
  • Propose to do this via the tvm-bot
  • Parse the body of an issue into potential labels
  • Want to encourage more longitudinal tracking
    • One of the ways currently established is the CI monitoring rotation schedule
    • With a dedicated monitor we end up with a list of flaky test issues
    • Want to make it possible for either the issue to be self triaged or
  • One option: Issue templates- downside was there were a lot of templates given the many components in TVM
  • Proposed option: Bug report template, with additional tags that the automated bot can discover and apply the correct labeling
  • No way currently to triage a PR and assign labels without committer status. By encouraging users to state the issue in terms of specific labels, the tvm bot can auto assign labels (triage) and cc relevant parties.
  • Another motivation: If you contribute code to TVM, there is a quick way to know who to attend
  • New labels!
    • Label group: BYOC (bring your own codegen)
    • Label group: backend: cover codegen and runtime of those backends
    • Label group: core, for core components of TVM that cut across runtime and compiler
    • Label group: dev, things that impact developer experience
    • Label group: flow, different compilation flows in TVM (relay, TIR, executors)
    • Frontend : importers
    • Relay : different sections of relay
    • Runtime
    • Topi : op implementations and scheedules
    • TIR
    • Tuning: the various tuning technologies
    • Vert: (verticals), issues facing a specific hardware targets end to end support
  • Deprecating labels:
    • Status, type, priority
  • Matthew Barret: Do we want to formalize the notion of calling things BYOC?
    • The difference between these we are calling byoc are that they are essentially Relay backends, whereas everything under “backend” is a TIR backend.
    • We currently map BYOC:relay backend, backend:TIR backend, is that the right distinction to formalize in the use of labels.
  • Gustavo Romero:
    • Wondering about the relation between the labels and the tags to be added to a title
    • Andrew: Current proposal (debatable) is to add a triage section at the end of the issue with the necessary labels,
      • Right now the title tags are user defined convention, with the new labels is one concern is the extra typing overhead. Main reason to propose additional labels that are prefilled in triage section is so that the user can just click and remove or add and not require more typing
    • How should we thread this through all the way to the PR commit message guidelines. Requires some thought.
      • Gustavo: Not apparently clashing, but having a set of tags to be used is part of the PR commit guidelines, and so having uniformity with triage issues. Multiple PRs may result from an issue so it doesn’t need to be a one-to-one map, but a uniform tagging/labeling system can provide helpful hints to where the PR derived from, especially if tags don’t immediately match with issue labels.