[Memory leak]How to free memory manually after building a operator for long running building case?

Questions
1

Hello all, I have a sample code from tvm offical site below. I want to build thousands of operators in the same process. But i found there is memory leak and the memory usage is increasing. I thought after the build of one operator was finished, the corresponding resource should be freed. Do you have any advice? Can i free memory manually after building a operator? Thanks a lot in advance!

Sample code(env: tvm 0.6, ubuntu 18.04, python3.7.5):

` import os, psutil import tvm

def tvm_build(): n = 2 A = tvm.placeholder((n,), name=‘A’) B = tvm.placeholder((n,), name=‘B’) C = tvm.compute(A.shape, lambda *i: A(*i) + B(*i), name=‘C’) s = tvm.create_schedule(C.op) m = tvm.lower(s, [A, B, C], name=“test_add”) rt_mod = tvm.build(m, target=“llvm”)

def main(): process = psutil.Process(os.getpid()) for i in range(0, 1000): tvm_build() print(‘i=’+str(i)+’, Used Memory:’, process.memory_info().rss / 1024 / 1024, ‘MB’) `

2

Running the code on the current master seems to work fine. Given that we are going to land another release, this problem may not appear in v0.7

Notably, there are constant memory used by global singletons which will always stay the same.

import os, psutil
import tvm
from tvm import te

def tvm_build():
    n = 2
    A = te.placeholder((n,), name='A')
    B = te.placeholder((n,), name='B')
    C = te.compute(A.shape, lambda *i: A(*i) + B(*i), name='C')
    s = te.create_schedule(C.op)
    m = tvm.driver.lower(s, [A, B, C], name="test_add")
    rt_mod = tvm.driver.build(m, target="llvm")

def main():
    process = psutil.Process(os.getpid())
    for i in range(0, 1000):
        tvm_build()
        print('i='+str(i)+', Used Memory:', process.memory_info().rss / 1024 / 1024, 'MB')


main()
3

Hi tqchen,

Thank you for your reply!

We are curently using the tvm v0.6. So is there any way to reduce the memory usage after building a operator for tvm v0.6? Can i call a method to free the memory? Thank you in advance!

BTW, I have tested these two sample codes for tvm v0.7 and v0.6. From the result, we can see that memory usage in tvm v0.7 version increased very slower than the one for tvm v0.6. That’s good, thank you for your effort!

environment: ubuntu v18.04, python v3.7.5

Test result for tvm v0.7(current master version):

Used Memory before building: 70.76171875 MB

Used Memory after building the first operator: 88.33984375 MB

Used Memory after building the last operator: 89.859375 MB

Used Memory after building: 89.859375 MB

Test result for tvm v0.6:

Used Memory before building: 80.60546875 MB

Used Memory after building the first operator: 93.828125 MB

Used Memory after building the last operator: 126.45703125 MB

Used Memory after building: 126.45703125 MB

Sample code for tvm v0.7:

import os, psutil
import tvm
from tvm import te

def tvm_build():
    n = 2
    A = te.placeholder((n,), name='A')
    B = te.placeholder((n,), name='B')
    C = te.compute(A.shape, lambda *i: A(*i) + B(*i), name='C')
    s = te.create_schedule(C.op)
    m = tvm.driver.lower(s, [A, B, C], name="test_add")
    rt_mod = tvm.driver.build(m, target="llvm")

def main():
    process = psutil.Process(os.getpid())
    print('Used Memory before building:', process.memory_info().rss / 1024 / 1024, 'MB')

    total = 5000
    for i in range(0, total+1):
        tvm_build()
        if i == 0:
            print('Used Memory after building the first operator:', process.memory_info().rss / 1024 / 1024, 'MB')
        if i == total:
            print('Used Memory after building the last operator:', process.memory_info().rss / 1024 / 1024, 'MB')

    print('Used Memory after building:', process.memory_info().rss / 1024 / 1024, 'MB')

if __name__ == "__main__":
    main()

Sample code for tvm v0.6:

import sys, os, psutil

def tvm_build():
    n = 2
    A = tvm.placeholder((n,), name='A')
    B = tvm.placeholder((n,), name='B')
    C = tvm.compute(A.shape, lambda *i: A(*i) + B(*i), name='C')
    s = tvm.create_schedule(C.op)
    m = tvm.lower(s, [A, B, C], name="test_add")
    rt_mod = tvm.build(m, target="llvm")

def main():
    process = psutil.Process(os.getpid())
    print('Used Memory before building:', process.memory_info().rss / 1024 / 1024, 'MB')

    total = 5000
    for i in range(0, total+1):
        tvm_build()
        if i == 0:
            print('Used Memory after building the first operator:', process.memory_info().rss / 1024 / 1024, 'MB')
        elif i == total:
            print('Used Memory after building the last operator:', process.memory_info().rss / 1024 / 1024, 'MB')

    print('Used Memory after building:', process.memory_info().rss / 1024 / 1024, 'MB')

if __name__ == "__main__":
    main()