Adding Data to a Relay Graph created with tvm.relay.fromtext

Questions
1

Hello,

I am trying to debug some issues I see with quantization in the compiler chain I am trying to build. To simpify testing I want to import a really basic test, which is easiest with fromtext, something like this:

mod = tvm.relay.fromtext("""
#[version = "0.0.5"]
def @main(%layer1_input: Tensor[(1, 16, 16, 16), uint8] /* span=aten::quantize_per_tensor_0.layer1_input:0:0 */, %c1_weight: Tensor[(16, 16, 3, 3), int8] /* span=quantized::conv2d_0:0:0 ) {
  %0 = qnn.conv2d(%0, %1, 0 /* span=quantized::conv2d_0:0:0 */, 0 /* span=quantized::conv2d_0:0:0 */, 1f /* span=quantized::conv2d_0:0:0 */, 1f /* span=quantized::conv2d_0:0:0 */, padding=[0, 0, 0, 0], channels=16, kernel_size=[3, 3], out_dtype="int32") /* span=quantized::conv2d_0:0:0 */;
}
""")

I want to start very small with zero_point = 0 and scale = 1. Is there a way to back this with data? For starters, a way to just set every value in the input and kernel to one would be good enough for me. Thanks in advance!

2

There are two ways to pass arrays into the Relay graph, either pass them as inputs, or embed them into the graph as constants. For the former

a = tmv.nd.array(numpy_array, device=...)

then pass a to the graph via set_input (usually—it depends on the executor).

If you want to embed it into the graph, create a “metadata” table:

mod_meta_table = {
    "relay.Constant": [relay.const(np.ones((16, 16, 3, 3), dtype=np.int8)]
}

pass it to the parser:

mod = tvm.relay.parse(mod_text, init_meta_table=mod_meta_table)

Refer to it in your Relay text as meta[relay.Constant][0].

What follows "relay.Constant" in the mod_meta_table is a list of all constants. The [0] in meta[relay.Constant][0] is the index of the constant you want to use.