- Will we allow composite functions to be treated as ‘ops’ for the annotation? This is necessary for the case where a series of operators are supported but the operators on their own are not.
The composite functions will not be treated as ops for op-based annotation, because all ops have to be registered in advance. We may implement op-based and function-based annotations in the same pass, and let function-based part deal with composite functions.
- I’m interested to know the use case for custom annotation at this point. If we got rid of it, we could just directly call the check function in the graph partitioner which would be a lot simpler.
We will keep custom annotation as an option for developers to make sure the annotation mechanism is capable of covering all possibilities.
- Do you think we could perform all the heterogeneous partitioning in this pass (e.g., GPU/CPU offload)?
I think that’s doable. @zhiics do you have more to comment?
I also wonder whether it might be appropriate to split the lowering into two phases. The first phase will just lower the graph in a way that conserves information and this is where you can insert your annotation passes. The second phase will be after partitioning where backend specific lowering is done depending on where the subgraphs were offloaded. All of the passes that destroy information can be moved after partitioning (like the ‘combine_parallel’ passes).
Although this proposal seems reasonable to me, I’ll pass this to zhiics as he worked on the pass manager.