How could I save the "graph", "lib", and "params"

zzw · April 24, 2019, 1:17pm

Through the following codes, we can get the “graph”, “lib”, and “params”:

graph, lib, params = relay.build_module.build(
net, target, params=params)

I want to save these at the local PC, and then copy to the remote machine to run the model for inferrence.

SrivastavaKshitij · April 24, 2019, 1:19pm

You could use pickle

zzw · April 24, 2019, 1:44pm

pickle only saves the “graph” file, but the “lib” and “params” can not be saved?

SrivastavaKshitij · April 24, 2019, 2:09pm

TVM Module class has a save function . check here (https://docs.tvm.ai/api/python/module.html), so your lib can be saved in this way

Params is a dictionary and can still be saved as pickle file

zzw · April 24, 2019, 2:31pm

Thanks for your suggestion.

srkreddy1238 · April 26, 2019, 4:55am

Pickle is good to save and reload again in python. But TVM has it’s own format for params.

Refer below to save params in a format accepted by TVM runtime while deployment.

github.com

dmlc/tvm/blob/6a956fbca3de973276158d0405dc2a458572f392/tutorials/relay_quick_start.py#L140


####################################################


# save the graph, lib and params into separate files
from tvm.contrib import util


temp = util.tempdir()
path_lib = temp.relpath("deploy_lib.tar")
lib.export_library(path_lib)
with open(temp.relpath("deploy_graph.json"), "w") as fo:
fo.write(graph)
with open(temp.relpath("deploy_param.params"), "wb") as fo:
fo.write(relay.save_param_dict(params))
print(temp.listdir())


####################################################


# load the module back.
loaded_json = open(temp.relpath("deploy_graph.json")).read()
loaded_lib = tvm.module.load(path_lib)
loaded_params = bytearray(open(temp.relpath("deploy_param.params"), "rb").read())
input_data = tvm.nd.array(np.random.uniform(size=data_shape).astype("float32"))

zzw · April 26, 2019, 8:08am

Thanks for your suggestion.

Ken · August 23, 2020, 10:24pm

Use relay.load_param_dict in python

leandron · August 24, 2020, 2:11pm

Saving your graph, lib and params is something covered by our ongoing PR around tvmc compile CLI driver: https://github.com/apache/incubator-tvm/pull/6302.