Not sure if I fully catch the point but let me try.
From the previous JSON runtime example I posted, you can see that the serialization mechanism can be fully customized, as long as your LoadFromBinary can construct an identical runtime module from the binary generated by SaveToBinary. In other words, although the file extension is .so, you can actually use any format of byte array.
We recommend the above approach, because users will only get a single .so file that includes everything – the TVM host module and your custom runtime module. Please note that we recommend this way even you are offloading an entire model (in this case the TVM host module has only one giant node), because it is general to cover both full and partial offloading.
Besides, you can still upload the .so file via RPC to a remote target. As long as your remote target has TVM runtime deployed, it should also include the implementation of your custom runtime, meaning that it can deserialize the .so file and build the runtime module.