[DISCUSS] Combine multi SYSTEM-LIB module libs to one

Questions
1

Deploy models on Android & iOS platform must use system-lib. Avoid usage of dlopen.
bundle_deploy demonstrated how to deploy a model which build target contains system-lib. Usually we deploy more than one model which tvm not support.

Working workaround: How to deploy two different tvm compiled model in c++ statically?
But this workaround has some limitations. AlterOpLayout pass must be disabled. AlterOpLayout pass can double the inference speed of some model.

Therefore I want to add Combine multi SYSTEM-LIB module libs to one function to tvm .
Work flow:

  1. Build each models separately
  2. Rename duplicate node name in all model’s GraphRuntimeFactory(json node name, params key, function name in lib), re-generate llvm module
  3. Generate modified graph json, params and a all in one lib
  4. Use runtime.SystemLib to load all in one lib
  5. Use tvm.graph_runtime.create & load_params to create each model’s runtime separately
2 Likes