Compile Pytorch model for iOS

Questions
5

Hello @L1onKing!

My colleague is working now on upstreaming some patches for ios support and ios-rpc, so I’ll ask him, and maybe he will be able to extend my answer with some additional details.

First, lets start from your script for building the model. You should do export_library differently:

  1. Import xcode utility: from tvm.contrib import xcode
  2. Use xcode.create_dylib in export_library:
    arch = "arm64"
sdk = "iphoneos"
libpath = model_folder_path + "/" + model_name + ".dylib"
lib.export_library(libpath, xcode.create_dylib, arch=arch, sdk=sdk)
  3. It is not necessary to dump your graph_json and params to a separate files.
  4. Add rpath to the library. Run in the terminal: install_name_tool -id @rpath/<model_name>.dylib <model_name>.dylib

After these changes your model should be ready to run it on a device.

Second, we need to compile TVM runtime for iOS:

  1. Use the following cmake flags:

    -DUSE_GRAPH_RUNTIME_DEBUG=ON # if you are interested in per layer performance statistic 
-DUSE_LLVM=OFF 
-DCMAKE_SYSTEM_NAME=iOS
-DCMAKE_OSX_ARCHITECTURES=arm64
-DCMAKE_OSX_DEPLOYMENT_TARGET=11.0  # compatibility with old version of iOS 
-DUSE_THREADS=OFF
-DCMAKE_BUILD_WITH_INSTALL_RPATH=TRUE # |
-DCMAKE_INSTALL_NAME_DIR="@rpath"     # | to make it portable without cmake install step

    You can change DCMAKE_OSX_DEPLOYMENT_TARGET to your target version.

  2. make -j<num_threads> tvm_runtime

  3. install_name_tool -id @rpath/libtvm_runtime.dylib libtvm_runtime.dylib

You can use these libraries in your iOS project.

2 Likes