[RFC] Standalone Code Generation and C Runtime for STM32 bare-metal devices

[manupa-arm]

I’ve been out for for Holidays and apologize for catching up on this a bit later. Thanks @stoa for the proposal and for the discussion.

I’m interested in the memory management aspect of the RFC here.

We propose to leave a full freedom of memory management to the main application (no TVM integrated memory manager). This will enable standard and familiar memory management techniques, such as using linker scripts, for example. Another existing project that follows this direction is the project to estimate the memory footprint of the graph from TVMC µTVM M2 Roadmap .

@areusch

When we do tensor pinning, I think it’s likely I’ll propose to add some tensor_id (note: different from storage_id, as storage_id could contain multiple tensor_id) to TVMBackendAllocWorkspace, and a lookup table could just return a pointer into the pre-allocated memory pool. TVMBackendFreeWorkspace would become a no-op. Will that work for you guys?

That is good. Just keep in mind that these memory pools should be open to a static allocation as a section via a link script, to a static allocation as a table from the main application (.data), and to the dynamic allocation via whatever allocator the application may choose.

Yeah this is all part of that. In particular, some accelerators may need a subset of parameters to live in a memory pool that lives at a fixed address for faster loading at startup.

@areusch @stoa based on the discussion that happened here, what is the current thinking as to who would be producing the address offset table in the case where user prefers TVM to figure out offsets from a single memory pool for all intermediary activations ?

Im looking at would there be an additional output to metadata.c/.o to hold the mapping between pinned tensors and their offsets.

Also @stoa, when you say dynamic allocation which granularity are we talking about ?

I mean would the application require to control the allocation of each and individual activation tensor or do you mean to say to decide where each pool (where all intermediary activations are allocated from) to be dynamically/statically allocated.

As a first step, we are starting to look at adding an interface to runtime.Module’s to be able to be queried for their workspace requirement to be consumed by the AoT executor initially. I will post a RFC soon. Is this something you guys have already looked at ?