ConstantNode variables mapping after the MergeComposite upgrade

Questions
#1

Hi all,

I found with the MergeComposite upgrade, it will take the model variables inside(like weights and bias) as ConstantNode in the codegen.cc rather than the former VarNode. This is more reasonable but I have some questions here:

  1. Where does the constant conversion happen? As if I print the mod after the partition I can still see the variables with names. And if I convert the call node into the function node, I can get the function node’s params with their name_hints. They are still VarNodes here.
  2. With the ConstantNode type, I cannot get the variable name anymore, how can I identify which variable it is? In the VisitExpr_ for ConstantNode here, we get the variables from the consts array, but how the order is decided?

Appreciate for any experts explaining for the ConstantNode processing here!

#2

I didn’t quite get your point. Maybe you could provide a minimal example to show how MergeComposite make your variables as constant nodes.

Based on my understanding, if your inputs are VarNode then MergeComposite won’t touch it, but if you apply bind_params_by_name to your model, then the bind parameters become ConstantNode. In this case, MergeComposite won’t create an argument but just simply access the constants from the pool.

For example you have the following graph:

%1 = nn.conv2d(%data, %weight)
%2 = nn.relu(%1)

After MergeComposte, you should get:

%1 = fn(%in1, %in2) {
    %a = nn.conv2d(%in1, %in2)
    nn.relu(%a) 
}
%1(%data, %weight)

If you bind the weight with constants, your graph becomes:

%1 = nn.conv2d(%data, relay.Constant[0])
%2 = nn.relu(%1)

After MergeComposite, you will get:

%1 = fn(%in1) {
    %a = nn.conv2d(%in1, relay.Constant[0])
    nn.relu(%a) 
}
%1(%data, %weight)
#3

Thanks for quick response!

I checked my code that, for example,

  • after from_onnx: %bert.encoder.layer.0.attention.self.query.bias
  • after bind_params_by_name: meta[relay.Constant][13]
  • after merge_composite: FunctionVar_2_7

This node in the graph is a constant while in the pattern I define is a variable, I guess after this matching and merge, it becomes the variable again. And when I get the variables in VisitExpr in my codegen.cc, it will be ConstantNode as the call->args[i]. Anything improper here? I define them as variables in the pattern to identify them easier.

auto res = VisitExpr(call->args[i]);

My purpose is to identify the res here, it’s fine if this call->args[i] is a variable so I can get the name_hint, but it is the ConstantNode.

#4

I think it’s because you use wildcard() in your pattern. In this case it matches both VarNode and ConstantNode, so the graph becomes:

%1 = fn(%in1, %in2) {
    %a = nn.conv2d(%in1, %in2)
    nn.relu(%a) 
}
%1(%data, %relay.Constant[0])

If you use is_constant() in your pattern, you will get the one I mentioned before

%1 = fn(%in1) {
    %a = nn.conv2d(%in1, relay.Constant[0])
    nn.relu(%a) 
}
%1(%data, %weight)

Further, if you use is_var() in your pattern, you won’t match any pattern with constant nodes.