Are web or webassembly supported autotvm

Questions
#1

tvm has supported rpc feature, but I cannot find RPCTracker for web. Do we have any suggestion?

#2

We dont need the rpc tracker for the web, but we do support a proxy in python that talks to the web via websocket and forward the connections to other places as described in the web folder

#3

Iā€™ve already build websocket demo, and run our model with web assembly backend in browser. but kernel is not tuned with this machine. Iā€™ve read some tutorials about android态cuda etc. I found rpc tracker is usefull.

Do we another way without rpc tracker that could be able to do tuning in browser?

#4

If we could run model in browser or any wasm environment by rpc connection. We could be able to measure the cost. Do we have convenience way to do these tasks ?

@tqchen any suggestion on the above issue.

Thanks

#5

We can use the rpc proxy with example-prc flag tvm/web at main Ā· apache/tvm Ā· GitHub

Which starts a RPC server via websocket and connect to the proxy. The proxy can then serve as a server(which can connects back to another trakcer if needed),

You can try to use the tracker process here tvm/rpc_proxy.py at main Ā· apache/tvm Ā· GitHub

#6

Thanks @tqchen . after below steps, I find a new problem, I donā€™t know how to send wasm_binary via rpc tracker. I passed session_constructor_args=["rpc.WasmSession", wasm_binary] in rpc.connect.

  1. launch rpc tracker

python -m tvm.exec.rpc_tracker --host=0.0.0.0 --port=9192

  1. launch rpc proxy and setup tracker address

python -m tvm.exec.rpc_proxy --example-rpc=1 --tracker=0.0.0.0:9192

  1. build a new rpc server as proxy way.

python -m tvm.exec.rpc_server --tracker=0.0.0.0:9192 --key=wasm --isproxy=1 --host=0.0.0.0 --port=9090

  1. python -m tvm.exec.query_rpc_tracker --host=0.0.0.0 --port=9192

127.0.0.1:62835 server:proxy[wasm]

  1. build connection

tracker = rpc.connect_tracker(ā€œ0.0.0.0ā€,9192)

#7

Iā€™ve solved the problem.

python -m tvm.exec.rpc_tracker --host=0.0.0.0 --port=9192

python -m tvm.exec.rpc_proxy --example-rpc=1 --tracker=0.0.0.0:9192

python -m tvm.exec.rpc_server --tracker=0.0.0.0:9192 --key=wasm-proxy --isproxy=1 --host=localhost --port=9090

Thanks.

#8

awesome, glad it works. I donā€™t think we have previous experiences tuning on web, would be great if you can share your findings as well

#9

Actually I havenā€™t completed turning on web. I still have jobs to make it work.

I think turning need two process.

  1. build network connection between web and local machine. so that I could upload and transform params or model and then run model on web, which have already worked.
  2. build the new model or module by localbuilder.

After turning, I found the library cannot run model on web. The error message is ā€œno such function in module: fused_layout_transform_42ā€. It looks like thereā€™s some incorrect in the process of building library.

Here is my code, firstly I use custom build_func to build wasm library.

builder=autotvm.LocalBuilder( n_parallel = 1, # build_func = ā€œwasmā€ build_func=fcompile )

def fcompile(*args): emcc.create_tvmjs_wasm(args[0], args[1]) fcompile.output_format = ā€œwasmā€

Durning turning I found that the library is smaller than these way

with tvm.transform.PassContext(opt_level=3): graph, lib, params = relay.build(mod, target, params=params) lib.export_library(wasm_path, emcc.create_tvmjs_wasm)

Any suggestion about that?

#10

Recently. I tuned a module which is not completed, but I found it speed up about 5% than fallback ways. It do need some hacks to make auto turning on browser. Iā€™ll share my findings after finished it.

1 Like