Can TVM split work into different layers, and assign layers into different cores?

[popojames]

Hello @hjiang

I have made some modifications based on your code and added CPU affinity setting into my function, but it seems to have some problems. I am looking for your advice.

According to the following previous posts,

1. Use all cores in a big.LITTLE architecture - #5 by FrozenGene

@FrozenGene

config_threadpool = remote.get_function('runtime.config_threadpool')
# affinity_mode: kBig = 1, kLittle = -1. kDefault = 0. pass 1 or -1 to control the cores
config_threadpool(affinity_mode, num_threads)

2. Setting the CPU affinity and number of cores locally without RPC Remote - #4 by popojames

Setting the CPU affinity and number of cores locally without RPC Remote

This problem is fixed by myslef. I was able to set it locally without RPC Remote. [image] What I did is very similar with the aforementioned setting: hooks c++ into python In my python/runtime/module.py, I add “conf_set” function inside, [image] to set the configuration, I can direct use this command just like we did in remote.get_function config_threadpool = module.module.config_set(remote.get_function('runtime.config_threadpool')

Now I am using config_threadpool of each subgraph to enable the CPU setting. In another word

1. Splitting the network into N subgraphs (Using your pipeline_graph function)
2. Then I add config_threadpool for each subgraphs
3. Then I assign CPU affinity to each subgraphs by config_threadpool_numofsubgraph(affinity_mode, num_threads)

For example, If I split into two sub-graphs wanna set first graph → 4 small cores and second graph ->4 big cores, I will use config_threadpool_0(-1, 4) and config_threadpool_1(1, 4) But it seems like those setting are not behave as I expect.

I am wondering how did you implement CPU affinity setting? Is it possible to share your code regarding how you set the cpu affinity even if it’s not fully ready to merge into main TVM?