[CMSIS-NN] qnn_max_pool2d layout on cortex M7 without NPU

Questions
#1

Hello, I am using CMSIS-NN for maxpooling on cortex M7 without Ethos-U and have the following questions:

  • Layout order : Maxpool supports HWC only. However if we feed qnn_max_pool2d with CHW layout, it does not complain and interpret the CHW parameters as HWC which leads to erroneous computation. (I did not see layout check/conversion into cmsisnn.py ) Is there a plan to add layout checking (and eventually layout conversion) ?

  • 1HWC Layout: Maxpool supports only N=1. However, if we give qnn_max_pool2d NHWC, it simply process the first batch without complaining. Is there a plan to support N to more than one on qnn_max_pool2d ?

Thank you very much for your help.

#2

cc @ashutosh-arm, @grant-arm who have been working on CMSIS-NN support

#3

Thanks @gilfr06 for bringing this up.

  • L1: It is possible to fallback to TVM schedule for maxpool when the layout is CHW since the Relay operator already contains this attribute (layout). However, this problem is more generic. Not all the operators that we use have the layout information available in Relay. Here, we assume in cmsisnn.py that that the network to begin with will have a layout of NHWC. We can fix this specific issue of maxpool. I am not sure about the process though: do we need to raise an issue on github for such requests? @Mousius @leandron ?

  • L2: I have asked internal team for more help with N>1 query. I should able to update about it soon.

#4

@lhutton1 @ashutosh-arm Thank you very much for your prompted responses.

#5

This is where @Mousius has fixed the first issue. https://github.com/apache/tvm/pull/11229.

#6

Thank you very much for having quickly fixed the checking part of CMSIS-NN pooling offloading.

#7

Raised the issue here in CMSIS-NN github: https://github.com/ARM-software/CMSIS_5/issues/1483 for N>1. Please feel free to add the reason for needing it @gilfr06 .

#8

@ashutosh-arm My use-case handles several samples at a time, which is supported by the different TVM operators.