Problem with AllocateConstNodes in CMSIS-NN code


I found another issue in the CMSIS-NN code. I experimented with a model that has both layers that get offloaded to CMSIS and some that do not. In that case AllocateConstNodes of CMSIS seem to get overwritten:

See these snippets from tvmgen_default___tvm_main__:

static const int32_t __attribute__((section(".rodata.tvm"), aligned(16))) constant_0[1] = {

if (tvmgen_mod_name_cmsis_nn_main_0(sid_1_let, constant_0, constant_1, constant_2, constant_3, constant_4, constant_5, sid_8_let, global_workspace_0_var) != 0 ) return -1;
if (tvmgen_default_fused_cast_subtract_clip_cast(sid_41_let, StatefulPartitionedCall_0_buffer_var, global_workspace_0_var) != 0 ) return -1;

with this implementation of tvmgen_mod_name_cmsis_nn_main_0

TVM_DLL int32_t tvmgen_mod_name_cmsis_nn_main_0(int8_t* input_, int8_t* filter_, int32_t* multiplier_, int32_t* filter_scale_, int32_t* bias_, int32_t* input_scale_, int32_t* shift_, int8_t* output_, uint8_t* global_workspace_2_var) {
  void* context_buffer_0_let = (&(global_workspace_2_var[160]));
  cmsis_nn_context context= {context_buffer_0_let,36};
  cmsis_nn_tile stride = {1,1};
  cmsis_nn_tile padding = {0,0};
  cmsis_nn_tile dilation = {1,1};
  cmsis_nn_activation activation = {-128,127};
  cmsis_nn_conv_params conv_params = {128, -128, stride, padding, dilation, activation};
  cmsis_nn_per_channel_quant_params quant_params = {multiplier_, shift_};
  cmsis_nn_dims input_dims = {1,50,1,3};
  cmsis_nn_dims filter_dims = {13,3,1,3};
  cmsis_nn_dims bias_dims = {1,1,1,13};
  cmsis_nn_dims output_dims = {1,48,1,13};
  arm_status status = arm_convolve_wrapper_s8(&context, &conv_params, &quant_params, &input_dims, input_, &filter_dims, filter_, &bias_dims, bias_, &output_dims, output_);
  if (status != ARM_MATH_SUCCESS) {
    return -1;
  return 0;

and this implementation of tvmgen_default_fused_cast_subtract_clip_cast

TVM_DLL int32_t tvmgen_default_fused_cast_subtract_clip_cast(int8_t* placeholder, uint8_t* T_cast, uint8_t* global_workspace_14_var) {
  for (int32_t ax1_inner = 0; ax1_inner < 2; ++ax1_inner) {
    int32_t _1 = ((int32_t)placeholder[ax1_inner]) - ((int32_t*)constant_0)[0];
    int32_t _2 = (_1) < (255) ? (_1) : (255);
    T_cast[ax1_inner] = ((uint8_t)((_2) > (0) ? (_2) : (0)));
  return 0;

Both tvmgen_mod_name_cmsis_nn_main_0 and tvmgen_default_fused_cast_subtract_clip_cast use constant_0 but clearly these should be different arrays. The datatype for arm_convolve_wrapper_s8 should be int8_t and the expected shape for the filter is (13,3,1,3) and not (1,).

Best, Sebastian @Khoi @MJKlaiber @MJKlaiberBosch @UlrikHjort @grant-arm @Mousius @manupa-arm @ashutosh-arm


Thanks for raising this @SebastianBoblestETAS . Is this a publicly available model that we could use to reproduce the issue ourselves?

Thanks for flagging this @SebastianBoblestETAS . Would you be able to raise an GH issue as well ?(hopefully with a way for us to reproduce)

Hi, thanks for the quick response. I will try to get one. Need to clarify this.

Yes, I will do that, thank you!

@SebastianBoblestETAS thanks for reporting this. It would be great if you can provide the model so we can reproduce it.

Hi @mehrdadh, see here: I attached a zip file there, that also includes a model that shows this behaviour.

I added a proposal for a fix here:

1 Like