An app can run two instances of the same NN model. In such situation, only one dll is loaded into memory, but two ModuleNodes are created, one for each instance. In CreateModuleFromLibrary(), the runtime modifies the dll’s content at symbol location runtime::symbol::tvm_module_ctx to make it point to the ModuleNode. Since we have only one copy of the dll, the pointer at runtime::symbol::tvm_module_ctx is overwritten by the later loaded model. Both models end up using the same ModuleNode created by the later loaded model.
For Vulkan runtime, this causes Vulkan error because the Vulkan pipeline for each shader is cached inside the ModuleNode. The two models end up sharing the same pipeline. When one model’s shader is running on the GPU, the other model tries to update the shader’s input.
I met this problem in Vulkan runtime. Not sure if any other runtimes have similar problem.