Some questions about vta runtime

Questions
1

Hi experts
I have some questions about vta runtime as show below.
How does the vta runtime separates the convolution operator from other operators, that is, how the vta runtime offloads the convolution operator to fpga, and puts the pooling or full-connect operators to the arm.
I go through the resnet tutorial and did not find the explicit offload operations in resnet-18.py, so it must be the underlying code to do this. So how should I look at the underlying runtime code correctly?

2

Hi there,

There are a number of threads in the forum about the VTA examples (granted mostly were in an older version, but I think the big picture has not changed) I suggest you search them for more info.

The VTA runtime does not select which operation is done on the FPGA.
I think the line of code which does this is here:

#This is just a copy past from the link!
@reg.register_compute("conv2d", level=15)
def compute_conv2d(attrs, inputs, out):
    """ 2D convolution algorithm.
    """
    padding = attrs.get_int_tuple("padding")
    strides = attrs.get_int_tuple("strides")
    dilation = attrs.get_int_tuple("dilation")
    groups = attrs.get_int("groups")
    layout = attrs["layout"]
    out_dtype = attrs['out_dtype']
    assert dilation == (1, 1), "not support dilate now"
    if is_packed_layout(layout):
        assert groups == 1
        return packed_conv2d(inputs[0], inputs[1],
                             padding, strides, out_dtype=out_dtype)
    return _nn.compute_conv2d(attrs, inputs, out)

Very briefly if is_packed_layout(layout): is the condition necessary for a conv2d to be off-loadable to the FPGA. Only NNVM Conv2d nodes which have this property are statically mapped to the VTA. This selection happens here.

3

Thank you very much for your help！ I am going to check the source code.