hello!
I know that TVM_NUM_THREAD is set for cpu affinity.
For example, Suppose the CPU has 8 cores.
In this case, I know that 2 processes set TVM_NUM_THREAD = 4 and TVM_BIND_THREAD = 0 to use 4 CPUs each.
If the above method is correct, what is the role of TVM_BIND_THREAD?
I think it is the number of CPUs that can be shared, but is this correct?
TVM_NUM_THREADS sets the total number of threads to use
TVM_BIND_THREADS enables or disables thread affinity.
I didnt find any docs, but this thread may help understand:
master ← apivovarov:affinity
opened 01:13AM - 18 Oct 19 UTC
TVM Runtime uses CPU Affinity by default.
It simply sets thread0 to use core0, … thread1 to use core1, etc.
Lets say we have 8 cores box.
User runs two TVM instances and sets number of threads to be 4.
`export TVM_NUM_THREADS=4`
Both TVM runtimes will use and share cores 0,1,2,3. Cores 4,5,6,7 will not be used.
As a result runtime performance will be 2 times slower.
I think CPU Affinity should not be enables by default. Most of the users are not aware of CPU Affinity in TVM.
If we disable CPU Affinity before running the test above then OS maps TVM runtime threads to particular cores itself. As a result each TVM runtime will gets its own 4 cores and all 8 available cores on the box will be utilized.
As a result two TVM runtimes will run in parallel without stepping on each other.
This PR disables CPU Affinity by default.
To enable CPU Affinity user needs explicitly set `TVM_BIND_THREADS` to be `1`
2 Likes