Alter_op_layout outputs 0 in shape

Questions
1

Hi,
I tried one script on alter_op_layout as follows. For the first layer, input is (1,3,224,244), infer type generates the layout_transform as ‘ty=Tensor[(1, 0, 224, 224, 4), float32]’

 @register_alter_op_layout("nn.conv2d", level=102)
 def alter_conv2d(attrs, inputs, tinfos):
   data, weight = inputs
   new_attrs = dict(attrs)
   new_attrs['data_layout'] = 'NCHW4c'
   new_attrs['kernel_layout'] = 'OIHW4o4i'
   return relay.nn.conv2d(data, weight, **new_attrs)

   x = relay.var("x", shape=(1, 3, 224, 224))
   weight = relay.var("weight")
   y = relay.nn.conv2d(x, weight, channels=64, kernel_size=(3, 3), padding=(1, 1))
   y = run_opt_pass(y, [transform.CanonicalizeOps(),
                     transform.AlterOpLayout()])
   print(y)

output:

free_var %x: Tensor[(1, 3, 224, 224), float32]
%0 = layout_transform(%x, src_layout="NCHW", dst_layout="NCHW4c") /* 
ty=Tensor[(1, 0, 224, 224, 4), float32] */;
free_var %weight: Tensor[(64, 3, 3, 3), float32]
%1 = layout_transform(%weight, src_layout="OIHW", dst_layout="OIHW4o4i") /* 
ty=Tensor[(16, 0, 3, 3, 4, 4), float32] */;
%2 = nn.conv2d(%0, %1, padding=[1, 1], channels=64, kernel_size=[3, 3], 
data_layout="NCHW4c", kernel_layout="OIHW4o4i") /* ty=Tensor[(1, 16, 224, 224, 4), float32] */;
layout_transform(%2, src_layout="NCHW4c", dst_layout="NCHW") /* ty=Tensor[(1, 64, 224, 224), float32] */

my question is:

  1. is ‘ty=Tensor[(1, 0, 224, 224, 4), float32]’ correct ?
  2. Need pre-process for input image to make it have shape (4, 224,224) ?

Thank you.

2

This is bug with alter_op_layout. Are you working on CUDA? You don’t need to pre-process. For non-divisible shape like (3, 224, 224), alter_op_layout will not be applied

3

Vinx13, Thanks for your help.
I’m not working on CUDA. I think of this case while I’m reading your PR in https://github.com/dmlc/tvm/pull/1735/files.

For non-divisible shape like (1, 3, 224, 224), test result shows it becomes ( 1, 0, 224, 224, 4). I’m not understanding what you mean by “alter_op_layout will not be applied”.

And if input image with shape (3, 224, 224) is not applied, then convolution compute in file conv2d_int8.py won’t be applied. What convolution compute does it use? Need specical handle for first layer ?

Best regards.

4

The layout is chosen based on tuning logs. Only if you have a NCHW4c layout in you logs, layout transform will be applied. If your input shape is (3, 224, 224), the operator in NCHW4c layout doesn’t work and you are unable to obtain the logs. So the layout transform will not be inserted

1 Like
5

I think we need to add a check for non-dividable layout transform