Hello! I would like to modify the operators in the Relay Intermediate Representation (IR) generated during TVM compilation in order to customize TVM compilation. I am referring to the method described in the following link: Adding an Operator to Relay — tvm 0.15.dev0 documentation However, I have a few questions about this approach. Firstly, the tutorial appears to add operators through Tensor Expression (TE) without directly modifying the Relay IR itself. Is there a way to directly modify the Relay IR? Secondly, when applying the tutorial, does the custom Relay operator override other operators during Relay transformation? I appreciate your assistance as always.
Any progress yet?
Hi, thanks for replying. Beside the previous question I asked, I’m currently working on manually creating a Convolutional layer with TE code compare the inference time between UINT8 and FP32 data types. To clarify, I haven’t made any progress with the question above. Thank you for your attention. If you’re interested, I’ve included a part of the code from above:
#-----------------------INT8 Basic example------------------------------------------------------------------------------
shape = (1, 3, 224, 224)
int8_input = np.random.randint(0, 255, size=shape, dtype=np.uint8)
dtype = "uint8"
data = relay.var("data", shape=shape, dtype=dtype)
kernel_shape = (64, 3, 7, 7)
kernel_dtype = "uint8"
kernel = relay.var("kernel", shape=kernel_shape, dtype=kernel_dtype)
conv2d_op = relay.nn.conv2d(data, kernel, padding=(3, 3, 3, 3), channels=64, kernel_size=(7, 7))
func = relay.Function([data, kernel], conv2d_op)
mod_int8 = tvm.IRModule.from_expr(func)
print("int8 Relay IR\n", mod_int8)
#-----------------------FP32 Basic example------------------------------------------------------------------------------
shape_1 = (1, 3, 224, 224)
float32_input = np.random.rand(1, 3, 224, 224).astype(np.float32)
dtype_1 = "float32"
data_1 = relay.var("data", shape=shape_1, dtype=dtype_1)
kernel_shape_1 = (64, 3, 7, 7)
kernel_1 = relay.var("kernel", shape=kernel_shape_1, dtype=dtype_1)
conv2d_op_1 = relay.nn.conv2d(data_1, kernel_1, padding=(3, 3, 3, 3), channels=64, kernel_size= (7, 7))
func_1 = relay.Function([data_1, kernel_1], conv2d_op_1)
mod_float32 = tvm.IRModule.from_expr(func_1)
print("float32 Relay IR\n",mod_float32)
#-----------------------check inference time----------------------------------------------------------------------------
print("floate32 optimization record\n=================================================================================")
inference_time_checker(mod_float32, {}, float32_input, "float32", "float32_tuning_record")
print("uint8 optimization record\n=================================================================================")
inference_time_checker(mod_int8, {}, int8_input, "int8", "int8_tuning_record")
Thanks again.