How to Add New Support for CMSIS-NN on TVM

#1

I hope to add CMSIS-NN support for PaddlePaddle models. Could you please tell me the general process? @ashutosh-arm @lhutton1

#2

TVM currently supports reading PaddlePaddle models. I would like to know how to enable operators to be successfully converted to CMSIS-NN operators. I saw in cmsis.py that qnn.conv2d is currently supported, but the convolution operator format in PaddlePaddle is (“qnn.dequantize”, “nn.conv2d”, “qnn.quantize”).

#3

I hope you can briefly describe the workflow of TVM for me. This will help me submit a PR more quickly to make the PaddlePaddle model compatible with CMSIS-NN. @ashutosh-arm @lhutton1

#4

Hi @Zheng-Bicheng, you can read more about the support for CMSIS-NN operators in https://github.com/apache/tvm-rfcs/blob/main/rfcs/0015_Arm_CMSIS-NN_Integration.md.

Currently all relay graphs that are offloaded to CMSIS-NN are captured in https://github.com/apache/tvm/blob/main/python/tvm/relay/op/contrib/cmsisnn.py, so here would be a good point to expand support for operators being converted to Relay by the PaddlePaddle frontend, and also to investigate, for example:

  • How to expand the frontend so that it generates Relay graphs that can be offloaded to CMSIS-NN OR
  • How to add patterns that will capture the ones being generated by the PaddlePaddle frontend

In both cases, for any change, I’d suggest you add lots of test cases, so that we guarantee that it don’t regress in future.

Hope that helps.

#5

Thank you for your reply, it’s very helpful for me. I will refer to your response to attempt to convert the PaddlePaddle model to CMSIS-NN.:grinning: