How to manually controll CPU affinity in multithreading scenario?

Questions
#1

Hello, I have a machine with 40 cpu cores, and a process that runs a TVM application on 10 threads. I wish that each TVM thread will use 4 cpu cores so I set TVM_NUM_THREADS=4.

Then what happens is without cpu affinity (TVM_BIND_THREADS=0), the 10 threads will try to occupy cpus randomly, slowing down on each thread.

I thought cpu affinity could help if we can let each thread have a dedicated cpu to use. e.g. thread_0 will use cpu_0~cpu_3, thread_1 will use cpu_4~cpu_7, etc.

I wonder if it’s possible to accomplish that, as when I set TVM_BIND_THREADS=1, all threads will try to use cpu_0~cpu_3. I wonder if there is a way to manually control the CPU affinity in a multithreading scenario. Thanks!

CPU affinity setting of pipeline process when using config_threadpool
#3

I also have the same question. Does anyone have any clue? Any pointers will be appreciated.

1 Like
#4

TVM doesn’t provide it currently.

1 Like
#5

Thanks @FrozenGene, is there a plan to add this in the future? I think this could be a useful function to add.

#6

@ZephyrSails I guess you can take look at Can TVM split work into different layers, and assign layers into different cores? - #10 by hjiang

I think this is what you are looking for.