FoldConstant doesn't fold two consecutive add

Questions
#1

Hi, I am using TVM to load an MXNet model and saw there is two consecutive add that do not get constant folded. Below are three screenshots showing (1) the Original MXNet model (2) the model without FoldConstant and FoldScaleAxis (3) with the relay transforms as below, which has two consecutive add

relay.transform.RemoveUnusedFunctions(),
relay.transform.CanonicalizeOps(),
relay.transform.InferType(), 
relay.transform.SimplifyInference(),
relay.transform.FoldConstant(),  # Commented out in (2)
relay.transform.FoldScaleAxis(), # Commented out in (2)
relay.transform.EliminateCommonSubexpr(),

(1) the Original MXNet model

image
image2750×804 127 KB

(2) the model without FoldConstant and FoldScaleAxis

relay.transform.RemoveUnusedFunctions(),
relay.transform.CanonicalizeOps(),
relay.transform.InferType(), 
relay.transform.SimplifyInference(),
relay.transform.EliminateCommonSubexpr(),

image
image2620×1712 303 KB

(3) with the relay transforms as below, which has two consecutive add

relay.transform.RemoveUnusedFunctions(),
relay.transform.CanonicalizeOps(),
relay.transform.InferType(), 
relay.transform.SimplifyInference(),
relay.transform.FoldConstant(),  # Commented out in (2)
relay.transform.FoldScaleAxis(), # Commented out in (2)
relay.transform.EliminateCommonSubexpr(),

image
image838×1014 16.9 KB

Some suggestions about how to fold two add will be awesome!

#2

If I understand your question, you want the two adds to fuse into one add. To do this, I would try using the FuseOps pass after FoldConstant and FoldScaleAxis. (Without seeing more of the program, I can’t tell what your two adds are adding – can you post an excerpt of the relay program here as well?)

#3

Hi @electriclilies thanks for the suggestion. I tried your method. However, I still see two adds instead of one. I will try to put on a code snippet later.

#4

你的这个问题解决了吗， 我也遇到了这样的情况

#5

This should have been supported since https://github.com/apache/tvm/pull/9671