Question about TVM-VTA deploy_resnet_on_vta.py

Questions
1

Hello, I’ve been looking at the source code of both tvm and vta for some time but it is still troubling me to understand the workflow beneath it.

Currently I’m running vta on pynq board.

From deploy_resnet_on_vta.py,

lib.save(temp.relpath("graphlib.o"))
remote.upload(temp.relpath("graphlib.o"))
lib = remote.load_module("graphlib.o")

# Graph runtime
m = graph_runtime.create(graph, lib, ctx)

I have two questions about above code.

First, I really want to understand what is contained inside the object code, and how to debug it.

Second, I want to know that I’m understanding the process correctly.

As I guess graphlib.o is a LLVM IR, cross compiled for arm. After remote.load_module, it returns a handle so graph_runtime is created for pynq board which can be invoked by functions like “run” in m.module.time_evaluator.

Thank you for reading and will be waiting for some answers :slight_smile:

2

There are a lot of layers to unfold to understand what happens under the hood that goes in the Relay to VTA compilation.

In terms of those lines, what we are doing here is compiling the operator implementations for each of the resnet layers as a library (.o file), which we upload on our remote (zynq) and load as a tvm module (https://docs.tvm.ai/api/python/module.html#tvm.module.Module).

Then the graph runtime takes a graph description of the relay program (think json based compute graph), and the tvm module lib to produce a TVM graph module (https://docs.tvm.ai/api/python/graph_runtime.html#tvm.contrib.graph_runtime.create).

To answer your questions:
(1) graphlib.o is the compiled LLVM IR cross compiled for arm that will call into the VTA runtime API functions (look at https://github.com/dmlc/tvm/blob/master/vta/include/vta/runtime.h).
(2) load module gives us a TVM module as previously said
(3) yes, m TVM graph module can let us invoke handy functions like time_evaluator to get std and mean stats of execution

Sounds like you have a pretty good handle on this already!

3 Likes
3

Thank you for responding with bunch of details. I really got a lot of help from reading your awesome papers :slight_smile:

1 Like