Can we create a standalone executable?

Questions
#1

Can we somehow create a standalone executable like glow and nnfusion?

https://github.com/pytorch/glow https://github.com/microsoft/nnfusion

Also, so far the compiled library seems to do not contain parameters, is it possible to embed the parameters into the binary?

#2

bundle_deploy

standalone excutable

standalone shared library

standalone static library

#3

@monkbai you can embed parameters by adding -link-params to your target string.

#4

thanks, I tried with bundle_deploy but ran into an error:

ls: cannot access '../../build/standalone_crt': No such file or directory
Makefile:24: *** "CRT not found. Ensure you have built the standalone_crt target and try again".  Stop.

The file “standalone_crt” does not appear to compile in the latest version of tvm. Do you have any suggestions to solve this problem?

#5

@areusch thank you, could you elaborate it further on how to add -link-params? I did not find an explanation in the doc.

#6

I solved it by setting set(USE_MICRO ON), it works well, thanks.

#7

@monkbai sorry for the delay, were you able to solve all of the problems you found? and, was USE_MICRO required for -link-params, or just to build the CRT?

#8

@areusch I successfully compiled the demo in the bundle_deploy folder by setting USE_MICRO, I still don’t know how to add -link-params. And USE_MICRO is just to build the CRT.

#9

hi @monkbai, you can add -link-params to your target string as explained here. for bundle_deploy, you can find this line in apps/bundle_deploy/build_model.py. you will also need to adjust the code in bundle.c to skip loading params from the JSON to truly cut out the dependency. if using bundle_deploy on a linux system, i’m not sure this is any better than loading from JSON, but you might see a speedup.