Great to see we’re getting there @areusch 
, though keen to get feedback from others.
Just to clarify where you’re heading here, I’m assuming the layers it’d go through are something like:
main.c
TVM_my_model_input inputs = {
.cats = &model_input_cats,
.dogs = &model_input_dogs
}; // Double input
TVM_my_model_output outputs = {&model_output}; // Single output
TVMExecute(&tvm_my_model, &inputs, &outputs, NULL);
tvm_micro_runtime.h
static inline int32_t TVMExecute(TVMModel* model, void* inputs, void* outputs, TVMContext* context) {
return model->entrypoint(inputs, outputs, context);
}
generated_entrypoint.c
static int32_t entrypoint(TVMModel_My_model_input* input, TVMModel_My_model_output* output, TVMContext* context) {
return tvm_my_model_run_func(input->cats, input->dogs, output->output_name, context);
}
Which seems neat enough, and I’m happier with having some types involved at the top level for users. I’m assuming we’d also want to include this in the model header file we’d generate which would make it slightly more than just metadata, but I would prefer that over several headers for a model.
Yip, I didn’t cover error handling in my example but the idea is just to propagate any returns from the AOT run_func back up the chain for the user to collect regardless of how that error is presented internally in TVM? If we need to check the actual error you can then use the other error functions?
Ah, gotcha, I think that’s best left up to the application developer as to where and how they initialise it.
Are you ok with deciding the exact level of internal organisation required at the code review stage for each incremental addition?
Revised Examples
Based on your feedback @areusch, I’ll provide some further examples of what I think we’ve discussed. I’ve used names for the inputs, outputs and workspaces, it’s worth considering whether we want to provide the facility for opting out of that for single input/output models simplicity?
Generated Model Header
Example for illustration:
tvm_my_model.h
// Input metadata
#define TVM_my_model_INPUT_CATS_SIZE 1234
typedef struct {
uint8_t* cats;
} TVM_my_model_input;
// Output metadata
#define TVM_my_model_OUTPUT_CLASS_SIZE 12
typedef struct {
int8_t* class;
} TVM_my_model_output;
// Model reference
extern const TVMModel my_model;
Default Model Compilation
#include "tvm_my_model.h" // Metadata, structs and extern definition (generated)
#include "tvm_micro_runtime.h" // Runtime interface (not generated)
uint8_t model_input_cats[TVM_my_model_INPUT_CATS_SIZE]; // User defined input buffer
int main() {
get_some_features_for_processing(&model_input_cats); // Fill the input buffer
int8_t model_output[TVM_my_model_OUTPUT_CLASS_SIZE]; // Output buffer
TVM_my_model_input inputs = {&model_input_cats}; // Single input
TVM_my_model_output outputs = {&model_output}; // Single output
TVMExecute(&tvm_my_model, &inputs, &outputs, NULL);
}
Custom-Workspace Compilation
#include "tvm_my_model.h" // Metadata, structs and extern definition (generated)
#include "tvm_micro_runtime.h" // Runtime interface (not generated)
uint8_t model_input_cats[TVM_my_model_INPUT_CATS_SIZE]; // User defined input buffer
uint8_t workspace[TVM_my_model_WORKSPACE_SRAM_SIZE]; // User created workspace of size specified by TVM
int main() {
TVMContext my_context; // Information for how to run in this environment
TVMSetWorkspaces(&my_context, {&workspace});
get_some_features_for_processing(&model_input_cats); // Fill the input buffer
int8_t model_output[TVM_my_model_OUTPUT_CLASS_SIZE]; // Output buffer
TVM_my_model_input inputs = {&model_input_cats}; // Single input
TVM_my_model_output outputs = {&model_output}; // Single output
TVMExecute(&tvm_my_model, &inputs, &outputs, &my_context);
}
Multiple Input Model
#include "tvm_my_model.h" // Metadata, structs and extern definition (generated)
#include "tvm_micro_runtime.h" // Runtime interface (not generated)
uint8_t model_input_cats[TVM_my_model_INPUT_CATS_SIZE]; // User defined first buffer
uint8_t model_input_dogs[TVM_my_model_INPUT_DOGS_SIZE]; // User defined second buffer
int main() {
get_some_features_for_processing(&model_input_cats); // Fill the first input buffer
get_some_more_features_for_processing(&model_input_dogs); // Fill the second input buffer
int8_t* model_output[TVM_my_model_OUTPUT_CLASS_SIZE]; // Output buffer
TVM_my_model_input inputs = {
.cats = &model_input_cats,
.dogs = &model_input_dogs
}; // Double input
TVM_my_model_output outputs = { .class = &model_output }; // Single output
TVMExecute(&tvm_my_model, &inputs, &outputs, NULL);
}