Hi there, I was trying to deploying with TVM with slight changes on model weights, however, it seems that the GraphModule always yields the same output even the params are updated.
The output keeps the same after new parameters are loaded. I have confirmed that the issue appear on both X86 and ARM arch. The TVM I am using is the based on commit 6720d3593d4dac6015418d4b7e9ad875bbf0b0a2 (submitted on Jan 24 2022)
This is probably because we embed (or bind, as it is called in the source code) all constant params into the mod itself. You can check the output graph, libs, params = relay.build(...) and check if params is empty.
Yes this means the params are hard-coded into the compiled lib. If you want to change them at runtime you need to make it an additional input (remove the param you want to change in relay.build(...) and do set_input). Note that this prevents compile-time computation on weight, so perf might get worse.
The load_params does not work properly because the keys are mapped to p0, p1, p2 … rather than the original name.
In my previous example, if change the key to
v = lib_params["p60"].numpy()
lib_params["p60"] = tvm.nd.array(np.zeros_like(v), tvm.cpu(0))
g.load_params(tvm.runtime.save_param_dict(lib_params))
g.run()
g.get_output(0).numpy()
# result all zero.