How can I static link C++ application with multiple TVM compiled models

Questions
#1

I see this topic here. he try to static link c++ application with model.o file and in the end this code help him.

However, in my case, my C++ application depend on many TVM compiled models (many model.so files) and I want to do static link instead of using model.so. How can I do it?

Is it still work if I use this code. If I don’t misunderstand this code only apply to 1 TVM compiled model.

Thank you very much!

P.S. I just see this topic How to deploy two different tvm compiled model in c++ statically?. However, I think it still on going question.

1 Like
#2

I’m looking for a solution to the same problem, but no luck so far.

This thread is also related: How can I change node name in tvm.Module

Do you guys any suggestion or can point to some code that I can edit to solve this issue? @tqchen @comaniac @vinx13

#3

Check apps/bundle_deploy.

#4

Thank you for your reply. I already check this (https://github.com/dmlc/tvm/tree/master/apps/bundle_deploy), but if I don’t misunderstand, it work only 1 compile model right? or Is it support 2 or 3 compiled models?

I see bundle_static.c, in tvm_runtime_create, the code have TVM_CCALL(TVMPackedFunc_InitGlobalFunc(&pf, "runtime.SystemLib", &args)); I still don’t understand how we select the model to use if we have multiple model.

Thanks again!!

#5

hi @juierror, you might take a look at [DISCUSS] Module based Model Runtime Interface

I’m not sure that RFC has been implemented yet, but I think that is the general idea. you might be able to do this by defining a Relay model with two model functions, but I haven’t tried this yet so i’m not sure what challenges you may face if you do this.

1 Like
#6

I believe you can mimic the call to register your own module with packed function to initialize. You’ll have to modify the runtime to identify which module your function belongs to though.

For the function registry, you will find this discussion useful: #7098

1 Like
#7

Thank you! I will check it.

#8

I see. I will check the link that you send to me. Thank you again!!