CPU affinity setting of pipeline process when using config_threadpool

[popojames]

Hello @FrozenGene @hjiang,

I have tried building pipeline executing on Hikey970 whcih contains 4 big cores and small cores. (Inspired by [RFC][Tracking Issue] Pipeline Executor For Compute graph pipeline · Issue #8596 · apache/tvm (github.com)) I am wondering if I have two processes running at the same time, is it possible to pin one of them on CPU 0-3 and another on CPU 4-7?

Here is what I did so far:

1. Split model into N subgraph (for example N = 2, I create mods[0],mods[1])
2. create by subgraph module 1 and module 2 by using graph_executor.GraphModule
3. Using multiprocessing to create the process for each module and set CPU affinity by using the following code

from tvm._ffi import get_global_func
config_threadpool_X = get_global_func('runtime.config_threadpool')
config_threadpool_1(-1, 4)  # In the process of module 1
config_threadpool_2(1, 4)  # In the process of module 2

However, it seems like config_threadpool would change the CPU affinity globally, which means that if I setconfig_threadpool on the process of module 1, it would affect another process. Can you please check that? If so, if I have two processes running at the same time, is it possible to pin one of them on CPU 0-3 and the other on CPU 4-7?

This post is a follow-up question of How to manually controll CPU affinity in multithreading scenario? ,and this question is also related to my previous question : Can TVM split work into different layers, and assign layers into different cores? - #30 by popojames

[hjiang]

@popojames , to set different cpu affinity to different runtime(module), user need to create different thread and call the config function within these different thread like what I mentioned in “Can tvm…” post, following is a example(included a example for Kspecify support in PR #9802).

you can set the break point in ‘t*.start()’ and using "taskset -cpa " to check the affinity changes.

import threading
import time
from tvm._ffi import get_global_func
config_threadpool = get_global_func('runtime.config_threadpool')

def test(name, affinity_type, cpu_list = []):
    print(f"Thread {name} start , type is {type} cpu_list is {cpu_list}")
    if affinity_type == -1:
        config_threadpool(-1, 4)
    elif affinity_type == 1:
        config_threadpool(1, 4)
    else:
        config_threadpool(-2, 4, cpu_list)

    time.sleep(200)
    print(f"Thread {name}: exit")

t1 = threading.Thread(target=test, args=(1, 1))
t2 = threading.Thread(target=test, args=(2, -1))
cpus1 = ['2','3','4','5']
cpus2 = ['6','7','8','9']
t3 = threading.Thread(target=test, args=(3, -2, cpus1))
t4 = threading.Thread(target=test, args=(3, -2, cpus2))
t1.start()
t2.start()
t3.start()
t4.start()