[Closed][RFC] Introduce unique module in TVM for uploading all deliverables into one

Development pre-RFC
#1

Hi All!

Introduction:

In this RFC, i like to propose an independent module inside TVM which will have key responsibility towards handling all the deliverables from TVM compiler(Relay Build) and offer an user friendly way to save, upload, reuse an TVM compiled library.

Motivation/Background:

Recently i was into developing one new runtime for TVM. Where accidentally i made a mistake while saving the output from relay.build(), basically library, graph & parameters. Which took me in a debugging path into some portion of code in TVM, anyway eventually i figured out what was the mistake. This lead me to the mentioned idea. My concern here is why user have to bother about certain file format, name format to save and reuse the deliverables from TVM.

So based on the background i stated above, i want to introduce one universal canonicalized file format for all the deliverables from TVM compilation irrespective of the target types. So in short there will be always one single file of one single format.

Responsibilities/Requirements:

So in order to achieve above mentioned goal, I have assigned certain responsibilities or requirements to the Unique module.

  1. File Package Management: Should have one in-built package manager which performs the task of uniting all deliverables into one single deliverable.
  2. Compression Support: Should be able to employ some lossless compression on top of it, to save some storage.
  3. Standardized Interfaces: To interact with TVM modules to perform saving and loading of various deliverables.
  4. Visualization Tool Support: This is optional.

Currently i can think of this much! So i am expecting on all TVM member expert opinions to fine tune above requirements or responsibilities.

I have a naive design based on above information, but i feel it is too early to state that here. First i think we all should brainstorm on below points:

  1. Whether the proposed module is helpful or required, so we pitch in to development/design.
  2. Curate all responsibilities under the module.
  3. Most important, find a name for the module :smiley:.
#2

Thanks for sharing the ideas. [DISCUSS] Module based Model Runtime Interface is the current proposed way for universal packaging, and that should resolve most of the current concerns.

While it is always possible to introduce another layer of abstraction for packaging, we also need to consider the minimum nature of the runtime and use as little abstraction as possible. i.e. the convention need to work on all platforms, languages and hopefully with as little dependency as possible (on ways to store libraries, availability of the dlopen etc).

Our current conclusion in the other thread is to reuse the native executable and dll format and try to package things into the shared library(so that it will always be available and backend indepedent). Feel free to chime on on that thread for more thoughts

#3

@tqchen: Thank you very much for your response!

I have gone through the RFC thread you shared. It has definitely addressed most of the concerns i mentioned here. @FrozenGene has done wonderful job there, and the discussions are really insightful.

However, i had a different solution in mind, i was planning for an independent module which will act as a bridge between Relay.Build and GraphRuntime (e.g., it consumes output from Relay Build and produce output to GraphRuntime). In that way we could achieve more control as well as abstraction. Which may open up to the areas like deliverables signing(security), ease on portability, compression and more importantly target independence(file formats) to user.

But as most of the pain-points addressed beautifully already, i want to experiment little more with the layout and code shared by @FrozenGene and yourself , before jumping to any conclusion.

Post that if i find some gap that can be filled, or some improvisation can be brought, will definitely share on the thread.

Thanks again!