# \[RFC\] A TVM command line interface **URL:** \ **Category:** RFC\ **Created:** [December 18, 2019, 4:23pm UTC](https://discuss.tvm.apache.org/t/rfc-a-tvm-command-line-interface/5165 "2019-12-18T16:23:42Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![mbaret](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/mbaret/32/1906_2.png) [@mbaret](https://discuss.tvm.apache.org/u/mbaret)\ **Post date:** [December 18, 2019, 4:23pm UTC](https://discuss.tvm.apache.org/t/rfc-a-tvm-command-line-interface/5165/1 "2019-12-18T16:23:42Z")
One of the challenges with using TVM as an ahead of time compiler is the lack of a simple interface to take a model and put it through the stack. The current solution of writing a python script has a few problems: - It leads to a lot of code duplication as everyone writes their own bit of python to accomplish the same task in a subtly different way. - It encourages writing special-cased solutions. - To even begin writing these python scripts requires a reasonably comprehensive knowledge of the API. The last point in particular makes TVM fairly inaccessible to anyone who is not interested in becoming a TVM developer. We have produced some python scripts to expose the core functionality of the TVM stack via a command line interface. ## tvm compile Compile takes a model in some format (eg. a .tflite file) and compiles it down to a tvm.Module which is saved to disk as a tar file (containing the graph/lib/params). Here we can set compiler flags and dump intermediates. ## tvm run Run takes a compiled tvm.Module (as a tar file) and runs it over RPC. Input tensors can be filled with zeros/ones/random or alternatively a .npz file can be passed to initialise the inputs. The outputs can then also be saved as a .npz file. This generic approach lets us run many different types of network instead of just image classification. Additionally, ‘run’ can time the execution and produce a profiling dump via the debug runtime. ## tvm tune Tune performs auto-tuning for a given model. This is the most speculative command as there exist a few special cases in tuning that are still best suited to a python script. However, the intention is to provide a ‘good enough’ route for someone who doesn’t want to develop an intimate understanding of AutoTVM. ## Usage A typical workflow would likely consist of the following: Perform some tuning `tvm tune resnet_v2_50.tflite -o history.log --host=123.45.6.78 --tracker-key=hikey960 --trials=1000 --target=llvm` Compile using the tuning history `tvm compile resnet_v2_50.tflite -o resnet_v2_50.tar --target=llvm --tuning-log=history.log` Run on a remote device `tvm run resnet_v2_50.tar --host=123.45.6.78 --tracker-key=hikey960` Or if you have an image converted into a .npz file `tvm run resnet_v2_50.tar --host=123.45.6.78 --tracker-key=hikey960 -i image.npz -o output.npz` And if you’re just interested in benchmarking `tvm run resnet_v2_50.tar --host=123.45.6.78 --tracker-key=hikey960 --time` ---
**Author:** ![haichen](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/haichen/32/246_2.png) [@haichen](https://discuss.tvm.apache.org/u/haichen)\ **Post date:** [December 18, 2019, 9:07pm UTC](https://discuss.tvm.apache.org/t/rfc-a-tvm-command-line-interface/5165/2 "2019-12-18T21:07:24Z")
Overall looks good to me. I think there’re a few additional flags that we should expose to CLI. 1. Allow user to specify the input shape mapping. 2. Allow user to specify context, e.g., gpu id. 3. AutoTVM allows user to specify a set of operators to be tuned. ---
**Author:** ![liangfu](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/liangfu/32/250_2.png) [@liangfu](https://discuss.tvm.apache.org/u/liangfu)\ **Post date:** [December 19, 2019, 2:37am UTC](https://discuss.tvm.apache.org/t/rfc-a-tvm-command-line-interface/5165/3 "2019-12-19T02:37:28Z")
Just want to make sure. Is this going to be a CLI in the python package? Maybe we can we run this via ```bash python3 -m tvm.cli tune ``` ---
**Author:** ![tqchen](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/tqchen/32/5_2.png) [@tqchen](https://discuss.tvm.apache.org/u/tqchen)\ **Post date:** [December 19, 2019, 4:42am UTC](https://discuss.tvm.apache.org/t/rfc-a-tvm-command-line-interface/5165/4 "2019-12-19T04:42:21Z")
As a side note, most of the current python CLI tools are under tvm.exec, ` python -m tvm.exec.rpc_server` ---
**Author:** ![comaniac](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/comaniac/32/899_2.png) [@comaniac](https://discuss.tvm.apache.org/u/comaniac)\ **Post date:** [December 19, 2019, 6:05pm UTC](https://discuss.tvm.apache.org/t/rfc-a-tvm-command-line-interface/5165/5 "2019-12-19T18:05:09Z")
Following this RFC, if we are going to have a Python CLI with the interface as @liangfu illustrated, should also move all current CLI tools to the top level (e.g., `python3 -m tvm rpc_server `)? It seems more general and reasonable in my opinion. ---
**Author:** ![tqchen](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/tqchen/32/5_2.png) [@tqchen](https://discuss.tvm.apache.org/u/tqchen)\ **Post date:** [December 19, 2019, 6:31pm UTC](https://discuss.tvm.apache.org/t/rfc-a-tvm-command-line-interface/5165/6 "2019-12-19T18:31:24Z")
Given that different cli tools have their own set of arguments, i think it is still better to distinguish them, in terms of cost of typing, both are quite similar. ---
**Author:** ![comaniac](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/comaniac/32/899_2.png) [@comaniac](https://discuss.tvm.apache.org/u/comaniac)\ **Post date:** [December 19, 2019, 6:44pm UTC](https://discuss.tvm.apache.org/t/rfc-a-tvm-command-line-interface/5165/7 "2019-12-19T18:44:31Z")
That’s a fair concern. FYI, a common way to distinguish commands with completely different functionalities is leveraging the hierarchical command groups such as `git`. For example: ```bash python3 -m tvm rpc server python3 -m tvm tune ``` We can use `argparse` with `add_subparsers` to achieve this. Based on that, we can also provide a decorator for modules to register their CLIs to a certain command group. Something like ```python @register_cli('rpc', 'server') def define_command(): parser = argparse.ArgumentParser(description='RPC Server') parser.add_argument('--port', default=9000, help='The assigned port') return parser ``` ---
**Author:** ![mbaret](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/mbaret/32/1906_2.png) [@mbaret](https://discuss.tvm.apache.org/u/mbaret)\ **Post date:** [December 19, 2019, 7:08pm UTC](https://discuss.tvm.apache.org/t/rfc-a-tvm-command-line-interface/5165/8 "2019-12-19T19:08:24Z")
Our implementation uses subparsers with ‘tvm’ being the principle command (maybe we should come up with a better name). I agree with using a hierarchical approach and that we could put additional scripts behind a unified CLI with their own appropriate subcommand. Ideally we’d have this CLI be installable with pip or similar. ---
**Author:** ![liangfu](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/liangfu/32/250_2.png) [@liangfu](https://discuss.tvm.apache.org/u/liangfu)\ **Post date:** [December 20, 2019, 1:41am UTC](https://discuss.tvm.apache.org/t/rfc-a-tvm-command-line-interface/5165/9 "2019-12-20T01:41:47Z")
I would agree with @comaniac. Hence, I would choose the verb “serve” as sub-command. This way we can use ```bash python3 -m tvm rpc serve ``` for RPC server, and use ```bash python3 -m tvm rpc connect ``` for RPC client. ---
**Author:** ![leandron](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/leandron/32/1205_2.png) [@leandron](https://discuss.tvm.apache.org/u/leandron)\ **Post date:** [July 22, 2020, 4:28pm UTC](https://discuss.tvm.apache.org/t/rfc-a-tvm-command-line-interface/5165/10 "2020-07-22T16:28:01Z")
Updating this discussion after a long time, today we submitted this PR: > > > TVMC - a command line driver for TVM > \* Introduce a command line driver to comp…ile, run and tune models, using TVM graph runtime > \* tvmc tests and integrate tvmc with regular linting, testing, CI and build scripts > \* RFC: https://discuss.tvm.ai/t/rfc-a-tvm-command-line-interface/5165 and more discussion at https://discuss.tvm.ai/t/usability-idea-create-a-command-line-compiler-that-would-perform-common-build-tasks-out-of-the-box/5129 ---
**Author:** ![tiandiao123](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/tiandiao123/32/1891_2.png) [@tiandiao123](https://discuss.tvm.apache.org/u/tiandiao123)\ **Post date:** [October 15, 2020, 10:34pm UTC](https://discuss.tvm.apache.org/t/rfc-a-tvm-command-line-interface/5165/11 "2020-10-15T22:34:50Z")
nice job!!! I am wondering how this command line know its input shapes if we don’t provide inout shapes and input names of those original models ---
**Author:** ![leandron](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/leandron/32/1205_2.png) [@leandron](https://discuss.tvm.apache.org/u/leandron)\ **Post date:** [October 21, 2020, 8:36am UTC](https://discuss.tvm.apache.org/t/rfc-a-tvm-command-line-interface/5165/12 "2020-10-21T08:36:19Z")
> [@tiandiao123](#): > > I am wondering how this command line know its input shapes Hi @tiandiao123, you can check [https://github.com/apache/incubator-tvm/blob/main/python/tvm/driver/tvmc/frontends.py](https://github.com/apache/incubator-tvm/blob/main/python/tvm/driver/tvmc/frontends.py), where all the input formats are dealt with. This is where we discover input shapes - each framework will have it’s own way. ---
**Author:** ![tiandiao123](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/tiandiao123/32/1891_2.png) [@tiandiao123](https://discuss.tvm.apache.org/u/tiandiao123)\ **Post date:** [October 22, 2020, 6:08am UTC](https://discuss.tvm.apache.org/t/rfc-a-tvm-command-line-interface/5165/13 "2020-10-22T06:08:33Z")
Thank you very much!!!