Hi,
I’ve started experimenting with TVM for a few weeks now. I would like to encapsulate the TVM runtime model loading and model execution into a WASM that can run in the browser. The purpose of this experiment is mainly for security, I want the ability to decrypt encrypted model files (graph, params, etc.) dynamically inside the Rust code without exposing the decryption key over network. Thus making it harder to “steal” the original model files.
To achieve this I was thinking to take a pre-compiled TVM model and wrap the model loading in Rust code that will later be compiled to WASM and exposed to the browser using predefined APIs.
To load the model files dynamically in Rust I was thinking of 3 options:
- Using
include_str!andinclude_bytes!to embed the model assets inside my WASM. - Fetch the model assets using an http client request from the Rust code.
- Fetch the model assets in JavaScript and pass it on to the WASM as input.
I was able to load the graph and params files dynamically:
let graph = include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/deploy_graph.json"));
let params = include_bytes!(concat!(env!("CARGO_MANIFEST_DIR"), "/deploy_param.params"));
My problem was instantiating the tvm runtime Module, as the API only accepts a Path to the lib and I was unable to utilize it using in-memory bytes.
let lib = tvm_rt::Module::load(&Path::new("../deploy_lib.so"));
This won’t work in WASM because there is no fs access.
Is there a workaround I can use to achieve the same result? I want to do something like this (doesn’t exist AFAIK):
let params = include_bytes!(concat!(env!("CARGO_MANIFEST_DIR"), "/deploy_lib.so"));
let lib = tvm_rt::Module::load_from_bytes(lib_bytes);
P.S I tried to achieve the same result with wasmtime - precompile a TVM model to WASM and load it dynamically in Rust code into memory and instantiate using wasmtime. I failed to compile this to WASM as wasmtime has some dependencies that cannot be compiled to WASM target.
Thanks, Barak