Setting per core usage explicitly in TVM

[johnmattew490]

Hi All,

I have a PC with 4 cores. I want to use 2 cores for the TVM inference and other two cores for another tasks in my application.

Example:
core 1 and core 2 == > TVM
core 3 and core 4 ==> My own task in my App.

I was able to set the to tell TVM to use 2 cores, but I was not able to figure out to explicitly bind specific cores to TVM. For example, I want TVM to use core 1 and core 2. However, it looks like TVM randomly allocates 2 cores among four cores when I start the inference. I have noticed that TVM inference sometimes chooses core 1 and core 2 and sometimes core 1 and core 3 …etc.

Any suggestions how to explicitly bind TVM inference to core 1 and core 2?

I also read that “TVM_BIND_THREADS =1” sets the CPU affinity, however, it seems it does not have any impact when I set it (TVM_BIND_THREADS=1) or unset it (TVM_BIND_THREADS=0).

[johnmattew490]

Hi All,

I found config_threadpool(“big”, 1) function which is in thread_pool.cc.

config_threadpool("big", 2) -supposed to changes affinity order to [4,5,1,2,3] , to make use CPUs 4, 5. However, I have a problem here. It sometimes uses CPUs 4, 5 and sometimes it uses 4, 6.

Any ideas? I want the TVM to always use CPUs 4 and 5. Please let me know if there are any suggestions.

[yidawang]

By default the runtime would bind the n TVM threads (specified in TVM_NUM_THREADS) to core 0 to n-1. TVM_BIND_THREADS=0 will instead rely on the OS to schedule the threads. However, the actual behavior may look similar with or without binding. In your case, I believe TVM_NUM_THREADS=2 TVM_BIND_THREADS=1 should work.

Specifying big is for ARM CPUs with big.LITTLE cores.

More info may be found here.