Is it possible to use the Python API to load models compiled by tvmc or use tvmc to run models saved by the Python API?
I followed the separate tutorials and noticed that there are similarities and differences in the outputs — for example the .so saved by the Python API appears to contain the same stuff as the mod.so plus mod.params produced by tvmc, but the Python API also needs some .o files which are absent with tvmc.
So is it possible to convert one output for use by the other?
@huanchunye it should be possible, yes. however, it depends on the format you use to export from tvmc (for example, Model Library Format is not meant to be loaded using load_module). could you give some more detail of what you’re trying to do?
@areusch We are using tvm for embedded devices. For security reasons we need to be able to update module + params separately from the code, so tvmc’s output formats are suitable. On the other hand we also use the Python API to integrate the tvm runtime with our system. I’d think this use case is common for embedded devices.
TVMC is a wrapper of python APIs. You can refer to python/tvm/driver/tvmc/runner.py for how to load it in python.
@huanchunye which runtime do you guys intend to use? If you’re able to use c++ runtime, then you should be able to easily accomplish this using the Python API (e.g. load_module, if your embedded device supports dlopen).
if you’re using the C runtime, things may be trickier. you could probably keep the compiled operators + params separate by placing them in a separate flash page, but you can generally not use the Python API directly with the C runtime (you can attach the C runtime to a Python interpreter using RPC, but I’m presuming this wouldn’t be your deployment scenario).