We have a huge module, after we import it to tvm, we can build it successful, also wo can run it correctly, but when I try to use export_library to export to run time library, there is an error like this:
I found some method to fixed it: add compile option “-mcmodel=large”, the same error happens. When I added this option “-nostdlib”, the error is different as the following:Any one can help me to solve this problem? Thanks.
Any one can give some suggestions?
are you using the llvm target? it might be that this option needs to also get passed down to the LLVM codegen in TVM. i’m not sure if we have this ability right now, but i think this would be the place to add it.
Are you using graph runtime or VM? It seems there has been a similar incident, and for VM it was apparently addressed in https://github.com/apache/tvm/pull/9734
Yes. Thanks, I will try it.
I am using graph runtime. Thanks for your information
Hi, @boh Have you solved this problem with graph runtime?
For bigger models, likely embedding weights into library is no longer a good idea. While there is not a solution with graph runtime, you can feel free to checkout latest solutions in TVM unity, e.g. MLC LLM | Home that lifts out the parameters from the module
