TVM-VTA Architecture Scope and Roadmap

Development pre-RFC
#1

Looking to get started on designing and organizing the documentation and developer walk-through and fully articulated use cases.

The first step is for the principals to provide a brain dump of the existing scope of TVM-VTA, the parts of the design space that were sampled, and the design decisions that were made to attack that scope with the existing design and roadmap.

The idea is that with this top level understanding of the scope and goals of the ‘program’ that individual contributors can then deep dive and generate good, contextual, documentation and cross references that will benefit information dissemination to the larger community.

#2

Usually, I find it would be good to answer specific items and open thread for each of them, like how does vta example of matmul work and what are the files that is involved. The general problem are good but hard to answer since that involve quite a lot of things(maybe everyone in the docs)

#3

Yep, that is the usual struggle between top-down vs bottom-up documentation. To be able to get a quick sense of the scope of the problem, and where you as an individual can contribute, requires an overview of the architecture and its major components.

I would like to know what are the major technology components of the full stack that have a ‘stand-alone’ quality: that is, they might be used in isolation. Clearly VTA is one such component, and I presume the IR can be such a component with IR writers, readers, and transformers.

What about the following components:

  • front-ends to TF, Pytorch, MXNET
  • the ‘compiler’ algorithms
  • are tensor ops stand-alone?

I do like the suggestion to make the walk-through very customer-facing. For VTA walking through an instruction is reasonable. What would be good ‘operators’ to show case for IR and its transformations? Are there opportunities to express an IR and show case the data structures and mappings to different back-ends to yield good efficiency (multi-core, DSP, GPU, OpenCL, FPGA, etc.)

#4

I’m all for improving our documentation to help the community navigate our stack, and understand how they could help improve it, and build new work on top of it.

Having a complete hardware software stack is rather new, and it might be nice to have a reference project/roadmap which we can try to emulate.

Are you looking for a quick top-down understanding of how VTA works from the model down to low-level instructions?

#5

actually, for me personally, the one thing I do understand is VTA. The stuff above it is less clear, :slight_smile:

I was looking for a high-level navigation of the technology pieces to enable divide and conquer for newbies like me to catch two birds with one stone, and that is, to take on a deep dive in one component and surface with both an understanding and a developer walk through/documentation.

The problem I am having is that I don’t know how the technology pieces are delineated and thus I can’t articulate a clear divide and conquer strategy to start building the detailed developer walk throughs.

2 Likes
#6

That makes sense, I think a complete overview to understand how the pieces work, would be nice. I’ll discuss this with my colleague @tqchen to see if we could write a blog post note by the end of the year that drills down through the stack.

In the meantime, if you have questions regarding how NNVM, TOPI, TVM, VTA, Relay all mean different things, I can do my best to try to explain.

1 Like
What is the relation among topi, nnvm, tvm, vta
#7

Love to make forward progress on this. The more documentation and working examples we have to push the VTA stack the quicker we can on-board new folks that want to add value here. Given the proliferation of DL hardware accelerators, this is a very fruitful avenue of research.

1 Like
#8

@Ravenwater certainly, I agree with your view on adding documentation and working examples, and drafting an agenda to make the community grow. I am currently on a long trip hence my prolonged absence, but will be back to work in a week or so. When I’m back in the swing of things, I’ll be happy to discuss with contributors how to best grow the VTA project moving forward in multiple directions.

Right now the main priorities I see are:

  • better front end integration with NNVM
  • multiple FPGA support
  • address scalability for data center FPGAs
  • support for novel data types (e.g. unums)

In general what needs improvements are:

  • more detailed, thorough documentation
  • extended tutorials

Please let me know if anyone would be interested in contributing to different aspects of this project.

1 Like