Exporting generated functions input and output sizes in the Model Library Format

microTVM
#1

Hi all,

I am creating this topic just as a placeholder to reference the description and use case of this pull request.

Current problem

The generated MLF header files for each module contain the struct definition to use as input and outputs to call the generated function. Some example (deleted comments to make it more readable):

struct tvmgen_default_inputs {
  void* input0;
};

struct tvmgen_default_outputs {
  void* output0;
  void* output1;
};

int32_t tvmgen_default_run(
  struct tvmgen_default_inputs* inputs,
  struct tvmgen_default_outputs* outputs
);

The problem is, if we want to call this tvmgen_default_run, we need to allocate space (statically or dynamically) for the input and output tensors. Although we have the entire io size as a macro, we dont have the specific size of each input and output.

Solution

To also generate macros that define the size of each input and output in bytes.

Example of the generated MLF header file with this change (deleted comments to make it more readable):

#define TVMGEN_DEFAULT_INPUT0 128
#define TVMGEN_DEFAULT_OUTPUT0 23
#define TVMGEN_DEFAULT_OUTPUT1 42

struct tvmgen_default_inputs {
  void* input0;
};

struct tvmgen_default_outputs {
  void* output0;
  void* output1;
};

int32_t tvmgen_default_run(
  struct tvmgen_default_inputs* inputs,
  struct tvmgen_default_outputs* outputs
);

This allows us to reference this new macros to statically or dynamically allocate vectors to store the inputs and outputs of the tvmgen_default_run function.

1 Like