[RFC] Optionally include object file generation in tvmc

Development pre-RFC
#1

Motivation

Currently tvmc will only produce a dynamic library version of the network, i.e., an .so file stored alongside the other artifacts. This library is usually dynamically linked to other applications.

With this change we want to add a flag to tvmc to get an object file (i.e., with a .o extension) as well as the dynamic library, to statically link against the target applications.

This will be very useful for bare-metal targets that cannot link against dynamic libraries.

Summary of the change

Standard behavior: tvmc compile mynetwork.tflite → mod.params, mod.so, mod.json

Enhanced behavior: tvmc compile --flag-name mynetwork.tflite → mod.params, mod.so, mod.o , mod.json

Possible names for the tvmc flag

Here is a list of possible flag names:

  • --bare-metal
  • --object-file
  • --static

Feel free to add more, and let’s agree on one in particular

1 Like
#2

cc: @leandron, @ramana-arm

#3

Hi,

Does --bare-metal imply that the C runtime has also been compiled and is part of the .o?

#4

Hi @aca88,

The object file produced by tvmc does not necessarily include the C runtime. Using a --bare-metal flag just refers to the fact that it is mostly useful on a bare-metal target.

Anyway, to avoid confusion, I think maybe --object-file might be a better choice :slight_smile:

#5

Ah sorry… I read that not as either or list :slight_smile:

#6

I think the idea is good, I remember it was raised on one of the code reviews by @FrozenGene, to fulfill this use-case of users who are not interested in tvmc run their modules, but instead, want to get an object file to link their own apps.

The question now is how do we name optional things into tvmc. I like --object-file, but sometimes I feel we should have an action before it, like --add-object-file or --enable-object-file.

cc @comaniac that has good suggestions about naming parameters.

#7

One thing to note is that the current TVM support this kind of behavior, by generating a library.tar, so if the output file format is specified to be .tar then it will output a tarball with a collection of object/c files that can be statically linked to a target app.

#8

Hi @tqchen,

tvmc saves directly the .so, .params and .json in the the .tar file it generates. This happens in tvmc/compiler.py. I might be wrong, but probably this is because it doesn’t want to store the .c files in the final artifact (@leandron, can you confirm this?).

#9

I was referring to the irmodule.export_library() API, where we can specify a tar. Perhaps we can try to unify the API terminology a bit. Note that under the new runtime packaging, we can directly package params json into dylib.

#10

From what I see, in tvmc.compiler, export_library() is called with a mod.so input.

I agree we could generate directly the tar file, but I think this was done to avoid storing the .c files (@leandron will know more than me on this).

As for storing directly in the dylib, I am not sure we can exploit this feature in a bare-metal flow. What I am after is the static object file (.o) to statically link with a main.c and the C-runtime (which will be subsequently executed on a bare-metal target)

#11

My code review is what TQ said. When we call export_library, we could save a.tar or a.so. If we save a.tar, which contains the object file (like a.o), this is different with tvmc's tar collections.

#12

I am also open to other proposals, the main goal here is to have a consistent API (between mod.export_library and tvmc)