[C++ runtime] only first lib is loaded when creating multiple c++ runtime

Questions
1

My c++ program is loading multiple TVM model the models are loaded with following code:

# readToString is a function to read file's content to memory as string.
auto graph_json = readToString("tvm_graph.json");
auto lib_dot_so = std::make_shared<tvm::runtime::Module>(
    tvm::runtime::Module::LoadFromFile("tvm_lib.so", "so"));
auto serialized_params = readToString("tvm_params.bin");

const int kDeviceId = 0;
sample_module = std::make_unique<tvm::runtime::Module>(
    static_cast<const tvm::runtime::Module&>(
        (*tvm::runtime::Registry::Get("tvm.graph_runtime.create"))(
            graph_json, *lib_dot_so, static_cast<int>(kDLCPU), kDeviceId)));
static_cast<tvm::runtime::GraphRuntime*>(sample_module->operator->())
    ->LoadParams(serialized_params);

after the model is loaded, I call them with:

auto gr = static_cast<tvm::runtime::GraphRuntime*>(sample_module->operator->());
gr->Run();

The strange thing is, when I do gr->Run();, only the first loaded model can be executed. If I run the second model, they will throw error like:

Check failed: pf != nullptr no such function in module: fused_nn_sdense_add

And fused_nn_sdense_add is a function exists only in the second model’s graph, not exist in the first model. So I suspect graph and params are correctly loaded for both models, but only the first model’s lib is loaded. But I don’t understand why the second model is trying to use the first model’s lib, maybe I’m loading them wrong? Thanks in advance.

1 Like