Measure Memory Allocation using Debug Executor


is there a way to measure the peak memory allocation of individual functions using the debug executor?

Right now there is not a way to collect this information. I’d like to add one, but it is a little bit complicated.

1 Like

I think @jwfromm may have a TIR pass for this, but it may be out-of-date.

1 Like

you may be able to use this recently-added analysis pass in combination with graph JSON information to compute the peak memory allocations. you’ll likely need to modify tvm.relay.build_module to call this–it may be tricky to use from pure TVM user code.

1 Like

Hi @max1996 ,

Have a go at this : Improved MLF to contain workspace info by manupa-arm · Pull Request #7938 · apache/tvm · GitHub

The metadata.json is augmented to include peak memory usage in the PR. Please note if you are using crt graph runtime, it might copy the weights (~ (if link-params is used && load_params is called) ) and also maintain a copy of I/O tensors in the workspace. For those reasons, that size is also mentioned in the .json.

Moreover, I’ve added instrumentation to see which relay primfunc and tir primfunc resulted in using that much of memory. In order to access this, in the PR, you could use…)'s output which is lib : GraphExecutorFactory. (Use lib.function_metadata).