APIs usage in BYOC

microTVM
#1

@areusch, Can you tell me which API’s to use and in what order to compile the newly exported codegen files (or say lib_xyz.so) instead of the tvm generated codegen source files?

I am aware of following APIs but I am not sure about their proper usage. I have tried experimenting with these APIs but I am still facing issues (no response during runtime) in a host-driven RPC based microTVM project:

c_mod = relay.build(mod, target=TARGET, runtime=RUNTIME, params=params)

with _make_session(project_dir, BOARD, "west", c_mod, aot_enabled) as session:
    location = "/tmp/"
    lib_path = location + "lib_xyz.so"
   
    c_mod.export_library(lib_path)
    comp_mod = session.get_system_lib()
    comp_mod = tvm.runtime.load_module(lib_path)

    graph_mod = tvm.micro.create_local_graph_executor(c_mod.get_graph_json(), comp_mod, session.device)

    graph_mod.set_input(**c_mod.get_params())

Can you guide me on how to properly use these APIs (and if I need additional APIs like import_module)?

#2

right now this looks pretty good to me aside from these two things:

    comp_mod = tvm.runtime.load_module(lib_path)

You should not need this line, and I’d expect it to do weird things. get_system_lib() provides a way to bypass load_module on an embedded device, and is necessary because we don’t assume the presence of a filesystem nor dlopen (both of which are used by load_module–this statement here in fact would attempt to load the model code into the local TVM runtime, rather than on the device). get_system_lib() is a function that runs on the device and returns the lone runtime::Module made available on the device.

    graph_mod.set_input(**c_mod.get_params())

You may try adding -link-params to an executor=Executor("graph", ...) passed to relay.build.

I suspect the lack of response means you’ve allocated too much RAM. -link-params would place those constant weights in flash. A stack trace/debug log would be helpful here as we could perhaps determine where the crash is occurring.