[RFC] [uTVM] Embedded C Runtime Interface

[Mousius]

@areusch

I think the struct TVMInputs_my_model approach is the better user-facing API. I would like to ensure that we can support a use cases which may implement some type of generic AOT driver function. Those use cases may prefer to operate in terms of void* inputs[] and void* outputs[]; I think these actually may be laid out in memory identically to TVMInputs_my_model, and so only a mapping from param_name → index is needed. If that turns out not to hold, we may need to map param_name → byte offset (into TVMInputs_my_model). I think this should be fine, just wanted to describe this use case to ensure you’re ok with it. It should not affect those applications which don’t want to use a generic driver.

I believe this should function as you’re intending and I do believe this is a valid use-case. My only concern is with relying on the structures matching the arrays memory layout but you’ve articulated a further workaround to that if required. If it becomes cumbersome, this is itself a thin layer over the AOT main and we could toggle a pointer array, or such generic use cases can instead use the packed API?

@areusch

I think this is fine. The exposed thing should include a pointer table, IMO, so that individual parameters could be overridden at runtime if necessary. wdyt?

I did envisage something like:

struct tvmgen_my_model_memory {
  void* sram, 
  void* flash,
  void* params
};

Where-as I believe you want to fine grain the parameters as well:

struct tvmgen_my_model_memory {
  void* sram, 
  void* flash,
  void* param0,
  void* param1
};

I don’t see an issue with breaking down the parameters further so we can set them for individual parts of the model. This could be useful for partial OTA updating

@areusch

Since we are likely preferring to emit LLVM object files, I do think we have to be somewhat concerned about sections and pass a degree of configuration up from LLVM API to the user. Section and alignment seem like reasonable parameters to expose. The use case I want to support is writing a script to generate a project based on an arbitrary MLF using some embedded platform.

I think you’d still require a linker script to move the relevant sections to the right memory locations, so it should be possible to generate a linker script to instead move specific symbols and enforce alignment?

@areusch

Yeah I think that’s fine–I meant more like struct-level \brief comments (e.g. for inputs: \brief Contains a pointer to the tensor data for each input to my_model)

I’ve put an example output in the PR, it has struct-level brief such as this, so hopefully that’s good