[RFC] Unified Static Memory Planning

[manupa-arm]

Hi @areusch

Thanks for taking time to read this!.

@areusch

I think currently @jroesch and @csullivan support S1 (see PR 7518, which my understanding says is still being worked on but which is often merge-conflicted). I also support this if it’s feasible to do so under all executors. I think the drawback is that non-AOT executors will need to run these passes, but the advantage is that it provides a clear framework under which we can consolidate post-scheduling whole-program modeling for both AOT and non-AOT use cases. Should we consider superseding VM executor with AOT in the future, it also provides a more natural pathway. I’m curious as to your opinions on this?

Yes, I do generally support the idea of having the whole program lowered to TIR. Im not sure about the VM and how important “static” memory planning is for VM. I think going forward graph executor might be able to load a packed function of the tvm_main instead of json – it’ll be less confusing as how the graph executor runtime is positioned as of today which is more of a (a very thin – as its supposed to be ) middleware that connect the graph json and the compiled operator library.

Having said that, I can see this work enabling a path (to extend) towards that – though we only plan to create USMP component that is a TIR IRModule → TIR IRModule which we initially test and support for the AoT executor.

@areusch

Just to confirm–would TVMContext also be generated e.g. TVMContext_my_model

Here I am referencing what is being discussed here : [RFC] [uTVM] Embedded C Runtime Interface - #6 by Mousius. I think its better to reach an agreement there. Here Im trying to illustrate and motivate the design using the APIs.

@areusch

I’m not certain the memory planner should necessarily encode all vars as buffer offsets–doing so could limit e.g. dynamic use cases, which may either a) need to express offsets as runtime-evaluated expressions or b) need to entirely defer such allocations to runtime, should it be impossible to pre-define such expressions.

This gets at my separation of concerns above–it would be nice to either

1. use the TIR-agnostic I/O format as a way to store the memory planner output and then inform further TIR modifications (e.g. either making everything buffer offsets when possible, passing those offsets in as positional arguments, or keeping TVMBAW for dynamic allocs)
2. represent that abstract output as e.g. TIR attributes and perform any of the aforementioned optimizations by examining TIR attributes

Yes, the USMP will only touch tir.allocates that could be evaluated in the compile time will be translated to offset. We could just leave the rest untouched to be TVMBAWs that is ultimately handled in the runtime (using malloc or stack_allocator). I think thats the only thing we require. Do we miss anything here? – I’ll adjust the original text to reflect this.

@areusch

In the tvmc command above, memory pools were identified by name. Any reason to translate to integers here?

No reason . Yes we could use the names – so its more clear.