Hi,
What is the exact difference between the operator types like ElemWise vs Injective, CommReduce vs OutEWiseFusable, and Tuple nodes?
1 Like
CommonReduce always means ReduceSum，ReduceAvg，as we know theses operators have reduce axis. Reduce Operator just reduce some axis to 1 value. As below shows j axis is gone.
reduce_sum(a[i, j, k], axis=j) = b[i, k]
Maybe you will say conv also reduce some axis, but it’s too complex, so it’s OutElwiseFusable. OutwiseFusable means after you calculate the op, you just add one more instriction to calculate next operator. One more instructions means elementwise op, like add, sub, relu.
I think the fusion rule defnition is for better writing schedule.
1 Like
Thank you @chenugray.
How an Injective op is different from elementwise op? any example can helps me.
Is it possible to fuse two or more Injective ops?
hi you can refer to the tvm paper which describes the fusion rule.
- injective + injective = injective
- injective + reduction = reduction
- OutElwiseFusble + element-wise = OutElwiseFusble