We observed that our App(which simply run TVM inference in a loop, using static input array to test latency), will always been killed(or crush) over night after long running. some investigation show the memory used by the App continuously increase, and we find a lot of unreachable memory allocations using dumpsys, and the number is increasing very quickly;
see dumpsys below:
dumpsys -t 600 meminfo --unreachable 17181
Applications Memory Usage (in Kilobytes):
Uptime: 502760754 Realtime: 502760754
** MEMINFO in pid 17181 [ml.dmlc.tvm.android.demo] **
Pss Private Private SwapPss Heap Heap Heap
Total Dirty Clean Dirty Size Alloc Free
------ ------ ------ ------ ------ ------ ------
Native Heap 33421 33328 4 54 59392 51006 8385
Dalvik Heap 8622 8600 4 53 8758 4411 4347
Dalvik Other 410 408 0 0
Stack 60 60 0 0
Ashmem 2 0 0 0
Gfx dev 236 236 0 0
Other dev 14 0 12 0
.so mmap 8479 256 6532 45
.jar mmap 4 0 4 0
.apk mmap 3454 2980 12 0
.ttf mmap 52 0 0 0
.dex mmap 2550 4 1232 0
.oat mmap 47 0 20 0
.art mmap 6737 6060 96 5
Other mmap 22 4 8 1
EGL mtrack 7720 7720 0 0
GL mtrack 3392 3392 0 0
Unknown 930 832 88 3
TOTAL 76313 63880 8012 161 68150 55417 12732
App Summary
Pss(KB)
------
Java Heap: 14756
Native Heap: 33328
Code: 11040
Stack: 60
Graphics: 11348
Private Other: 1360
System: 4421
TOTAL: 76313 TOTAL SWAP PSS: 161
Objects
Views: 8 ViewRootImpl: 1
AppContexts: 5 Activities: 1
Assets: 2 AssetManagers: 0
Local Binders: 8 Proxy Binders: 27
Parcel memory: 5 Parcel count: 20
Death Recipients: 1 OpenSSL Sockets: 0
WebViews: 0
SQL
MEMORY_USED: 0
PAGECACHE_OVERFLOW: 0 MALLOC_SIZE: 0
Unreachable memory
27643912 bytes in 398 unreachable allocations
ABI: 'arm64'
69632 bytes unreachable at 738601e000
first 20 bytes of contents:
738601e000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
738601e010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
69632 bytes unreachable at 738602f000
first 20 bytes of contents:
738602f000: 00 00 80 3f 00 00 80 3f 00 00 80 3f 00 00 80 3f ...?...?...?...?
738602f010: 00 00 80 3f 00 00 80 3f 00 00 80 3f 00 00 80 3f ...?...?...?...?
69632 bytes unreachable at 7386040000
first 20 bytes of contents:
7386040000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
7386040010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
....
I think the memory leak is due to this pointer assignment below:
here from->data
may already pointed to a pre-allocated memory space; Actually, any call for NDArray’s copyFrom()
method(see below), an temp NDArray tmpArr
will be created with pre-allocated memory, and the current implementation of tvmArrayCopyFromJArray
will cause this pre-allocated memory unreachable, due to the pointer assignment mentioned above;
I used memcpy(from->data, static_cast<void *>(data), size)
instead of from->data = static_cast<void *>(data)
; and verified the memory leak problem in my application has disappeared, however I’m not sure whether this is a generic solution for all cases;