FoldConstant doesn't fold two consecutive add

Hi, I am using TVM to load an MXNet model and saw there is two consecutive add that do not get constant folded. Below are three screenshots showing (1) the Original MXNet model (2) the model without FoldConstant and FoldScaleAxis (3) with the relay transforms as below, which has two consecutive add

relay.transform.RemoveUnusedFunctions(),
relay.transform.CanonicalizeOps(),
relay.transform.InferType(), 
relay.transform.SimplifyInference(),
relay.transform.FoldConstant(),  # Commented out in (2)
relay.transform.FoldScaleAxis(), # Commented out in (2)
relay.transform.EliminateCommonSubexpr(),

(1) the Original MXNet model

(2) the model without FoldConstant and FoldScaleAxis

relay.transform.RemoveUnusedFunctions(),
relay.transform.CanonicalizeOps(),
relay.transform.InferType(), 
relay.transform.SimplifyInference(),
relay.transform.EliminateCommonSubexpr(),

(3) with the relay transforms as below, which has two consecutive add

relay.transform.RemoveUnusedFunctions(),
relay.transform.CanonicalizeOps(),
relay.transform.InferType(), 
relay.transform.SimplifyInference(),
relay.transform.FoldConstant(),  # Commented out in (2)
relay.transform.FoldScaleAxis(), # Commented out in (2)
relay.transform.EliminateCommonSubexpr(),

Some suggestions about how to fold two add will be awesome!

If I understand your question, you want the two adds to fuse into one add. To do this, I would try using the FuseOps pass after FoldConstant and FoldScaleAxis. (Without seeing more of the program, I can’t tell what your two adds are adding – can you post an excerpt of the relay program here as well?)

Hi @electriclilies thanks for the suggestion. I tried your method. However, I still see two adds instead of one. I will try to put on a code snippet later.