In Relax Architecture Overview · tlc-pack/relax Wiki · GitHub , it says
" we start to see a strong need of performing optimizations across the layers. For example, ideally the automation decisions in TensorIR should inform fusion and layout decisions at the high-level."
How do the decisions in TensorIR affect the decisions in the high-level? Are there any more specific examples? After all, in the traditional one-shot compilation mode, the low-level cannot affect the high-level.
I’ve encountered some situations maybe related to this recently. For example, there have been some interesting optimization ideas like welder (osdi 23), which explores the potential and benefits of shared memory connect. However, whether it’s beneficial to connect several ops in shared memory or not, that should be investigated at a low-level, moreover, the results should also be reflected in high-level graph (to combine some operators). such an idea can be implemented more smoothly in relax.
tir can also help us to explore some sth low-level to guide high level optimizations, like deducing the layout based on the ast and domain knowledge, or automatically checking whether some ops are fusable I guess.
I’m very grateful for your response to my question. Your answers and the examples you’ve provided have significantly improved my comprehension of the topic at hand. Thank you!
These are all great examples of cross layer optimizations. With hardware becomes more specialized, we will see more need in this direction