Can TVM split work into different layers, and assign layers into different cores?

[popojames]

Hello @hjiang,

Thanks very much for sharing your implementation.

First of all, I wanna double-check if I wanna re-build a new TVM environment, should I go with this GitHub branch: GitHub - huajsj/tvm at threadaffinity?

---

Regarding your reply: as you explained in [Runtime][ThreadPool]Refactor affinity function and support CPU affinity list setting. by huajsj · Pull Request #9802 · apache/tvm · GitHub, you introduced “kSpecify” and “tvm::runtime::threading ::Configure” to specify the CPU list for the CPU affinity. For this, I have some general questions.

1. In this setting, I wanna double-check if I wanna use the “kSpecify” mode, is it similar to the original setting as shown below (i.e, calling config_threadpool) but with CPU affinity mode = -2 and adding parameters “cpus” and “exclude_worker0”?

@FrozenGene

config_threadpool = remote.get_function('runtime.config_threadpool')
# affinity_mode: kBig = 1, kLittle = -1. kDefault = 0. pass 1 or -1 to control the cores
config_threadpool(affinity_mode, num_threads)

2. For the example that I mentioned above, if I wanna split the network into two sub-graphs, then set the first graph → 4 small cores, second graph ->4 big cores. (1) Should I set “cpus” =[4,4]? (2) How exactly to set the small & big CPU affinity order?

3. I do not fully understand how to implement this instruction:

   
   How exactly to launch multi-threads if I call tvm backend in python to run the benchmark?

4. Is it possible to share any example code for this CPU affinity setting? I think one simple example like a hand-made multilayer perceptron with CPU splitting would be really helpful for me and other users to understand the whole process.

Thanks again for your great help. Happy new year