[DISCUSS] Module based Model Runtime Interface

Note: this is relatively independent from the problem of minimize device blob obj generation. Looking at the size of the parameters, two things comes to my mind:

  • First of all, if we are going to export the parameters with the DSO, whether or not should we offer the quick option to compress them in the binary(e.g. zlib the parameters before serialization, or at least offer it as a form of storage format.
  • We should have a simple way to allow user to specify not to include the parameters in the DSO, it would be interesting to ask what that API should look like

For example, here is one possibility. It might also worth ask everyone’s opinion about the API and thedefault option

mod.export_library("xx.so", package_params=False)