Operator fusion question

Questions
#1

As far as I know, TVM tries to fuse a matmul with a following element-wise operator, but what about the following two cases:

  • Matmul followed by a reduction op, such as softmax
  • Matmul followed by another matmul

Will TVM try to fuse that? Thanks!

#2

These cases are not fused. The fusion itself would be trivial, the challenge is scheduling (to generate efficient code).

1 Like