Looks like SimplifyExpr
doesn’t support folding bias_add
and bias
, see tvm/src/relay/transforms/simplify_expr.cc at 6942b3660df3551a3a9a86c2faba834d366a2a7e · apache/tvm · GitHub. So both of cases don’t work unless you modify that pass. But I recommend not depending on bias_add
as explained below.
I highly suggest modifying your pattern to support both bias_add
and add
as in tvm/python/tvm/relay/op/contrib/cutlass.py at 7fd73b2663ae33d341ab09834f215285eb9bd136 · apache/tvm · GitHub
Frontends are not consistent in which ops to generate for bias addition.
And I recommend experimenting with simple test cases like tvm/tests/python/relay/test_pass_fold_constant.py at ac6607282e080dc15cce7d9cf565f5d390ba0f16 · apache/tvm · GitHub rather than the whole resnet50 from the beginning.