Hello, tvmc provides the functionality to save a compiled model to a .tar file after calling the compile subcommand. Is there a way to do this though the TVM Python API after generating a tvm.module library via relay.build(mod, target, params)? Using the Python API, I want to be able to export the graph, parameters, and library in a TAR package before running inferences on the target hardware.
Thanks!
Hi @calebpsanders, you can also use tvmc as a Python API.
Have a look on what is described in https://github.com/apache/tvm/pull/7823
We don’t yet have a tutorial covering that specifically, but the code is quite clear and easy to understand.
Also cc @jwfromm
Hi @calebpsanders, just wanted to echo leandro and point you to this end-to-end test for the python tvmc api. It breaks down the full flow into 4 function calls. Sorry about there not being a tutorial yet, we’re working on it and will hopefully upstream it soon.
Awesome, this is exactly what I was looking for. Very easy to use and intuitive. Nice to see tuning customization still very well supported. Would be great to have some tutorials on this. Just read through the link that @leandron set over, looks like there might be a little hesitancy on the implementation of this moving forward? Should I expect many changes to this functionality in the future? Thanks for your help guys!
One more clarifying question, running tvmc.compile(model, target) and then model.save(path), saves the model compiled for a specific target, correct? This model can then be re-loaded loaded and either tuned or run on the target?
Glad you like the API @calebpsanders! We’ll continue to expand the capabilities of the API but are fully committed to maintaining it so no worries there. tvmc.compile actually produces a TVMCPackage object meant for deployment that is saved to the package_path argument. If you want to retune the model, you should save the original TVMCModel using model.save. Note that there are two distinct TVMC artifacts, the precompiled and tunable TVMCModel, and the ready to deploy TVMCPackage.
Ultimately, I’m trying to save off a tuned model as an executable (or something similar). I’m wondering if this should be done between the tune and compile steps, or between the compile and run steps. However, I’m a little confused on the difference between the TVMCModel and TVMCPackage artifacts. The only difference between these two is that the TVMCPackage artifact has a mod.so file while TVMCModel does not - does this .so file make the TVMCPackage deployable?
Yes TVMCPackage (between the compile and run steps) sounds like what you want. It’s the deployable artifact. TVMCModel is the high level graph definition. If you wanted to tune a model for a few different hardware targets you could use the one TVMCModel to produce several TVMCPackages for example.