Is memory scope in tvm is just for virtual concept?

Development
#1

hi ,
Recently i have been experimenting with memory_scope feature of tvm. Though it is great to have the flexibility to use intermediates mem_scopes to mimic working of different hardwares ,
I wanted to know if this is just a virtual concept , i.e is there a way to define these mem_scopes or map these mem_scopes to actual hardware memory ?
Origin of the question :
lets say ,
there are 2 input buffers , each goes through a shared memory .
Now in the generated schedule , there are 2 new buffers allocated for 2 input buffers with scope as shared.whatever.
Question : Since TVM is allocating 2 new buffers and forcing the computation to go through the external shared memory (for compilations purposes i believe) , but is there a way to tell tvm to tag the buffers with a memory scope and map the scope with a physical memory of a hardware instead of allocating the memory explicitly.

Also once we build this schedule and run it , it throws this warning :

 Warning: MemoryInfo for scope = shared.custom_mem is undefined

which made me wonder , can we actually map a memory section of hardware with this memory scope tag?
@tqchen @sanirudh .

#2

The memory scope can be used by the runtime to allocate the buffer in the correct location, and the way this is done is probably different for each target runtime.

The way this normally works is in TIR you would have an AllocateNode which gets lowered (in general) to TVMBackendAllocWorkspace, whose implementation by default is to call the AllocDataSpace function from device_api at runtime.

This flow can be altered at any point to replace the AllocateNode to some other builtin that a particular backend can understands (something like this is done for hexagon in this pass), and appropriately allocate in the correct shared memory with the target specific runtime device_api calls.

So yeah, to answer your question, the memory scope tag is to actually allocate to the correct memory at runtime, but for a new memory_scope, you would need to add support as needed in a few places.

1 Like
#3

@sanirudh thank you for crisp explanation. would try this .