Background:
I’ve been following the auto-tuning procedure detailed in this documentation link: Auto-tuning a Convolutional Network for ARM CPU — tvm 0.9.dev0 documentation
The LocalBuilder class used in that tutorial creates multiple BuildResult objects from which the RPCRunner class extracts the filename field, which contains the path to the temporary model .so file that is pushed to the device via RPC, that is ultimately run on the target device for time measurement.
My Problem:
My target device can’t support RPC communication, so I want to manually run the temporary model files by pushing the .so files to the device via adb, and running a custom, barebones executable that loads the model in the .so, and runs it.
In my custom app I’m following the pattern demonstrated here https://github.com/apache/tvm/blob/main/apps/howto_deploy/cpp_deploy.cc I’m able to load the .so successfully via the LoadFromFile API. However, when I try to run:
tvm::runtime::Module gmod = mod_factory.GetFunction("default")(dev);
I get a segfault.
My Question:
How do I run these temporary model .so files created by the LocalBuilder? Is there anything unique about these temporary models that I’m missing?
More Context:
- I’ve made sure that the target triplet is correct, and that the NDK toolchain path is properly exported. In fact, the target settings are identical to when I first compiled my model (without autotuning), and that ran fine on my target device using the GraphExecutor
- I noticed that the AutoTVM LocalBuilder uses tvm.driver.build, while the public documentation dictates using tvm.relay.build-- the latter of which I have used just fine. Perhaps this subtle difference is causing the issue?
- I’ve previously used the GraphExecutor class explicitly to run my models. However, the output of the LocalBuilder does not contain the params nor json files that are required to use the GraphExecutor explicitly.