Support for TVM Relay with WASM Runtime

videetparekh · May 21, 2020, 8:03pm

I have an int8 quantized model (with json, params) compiled to WASM and another params file that holds dequantization data. While ingesting the model in a NodeJS app, I need to dequantize the weights back to fp32, which requires me to read the parameter files, dequantize the weights and save the parameter dictionary again.

Is there a way to support TVM Relay functions that allow access to the graph and parameters(save_params_dict, load_params_dict) with a WASM library the way the TVM Runtime is supported?

videetparekh · May 26, 2020, 4:09pm

Following up here: Are there plans to extend TVM Relay support to WASM?

3160104812 · June 12, 2020, 12:57pm

Hi, I am kind of curious your method to compile the module,json,params items into 1 .wasm. May I get some guidance on that?

videetparekh · July 15, 2020, 8:46pm

Here’s a great example by Tianqi to compile to the model.

github.com

tqchen/tvm-webgpu-example/blob/master/build.py

"""Example script to create tvm wasm modules and deploy it."""

import argparse
import os
from tvm import relay
from tvm.contrib import util, emcc
from tvm import rpc
from tvm.contrib.download import download_testdata
import tvm
from tvm._ffi import libinfo
import shutil
import logging
import json

curr_dir = os.path.dirname(os.path.realpath(os.path.expanduser(__file__)))


def build_module(opts):
    network = opts.network
    build_dir = opts.out_dir

This file has been truncated. show original

I take this and apply it to a pretrained TF model to compile to WASM. If you look carefully at the repo, you will also find a sample script to deploy the model for WebGPU.