Type solver cant resolve custom Relay operator

I wrote a custom Relay operator for a custom codegen. It is a fusion of nn.conv2d and nn.bias_add. First I fuse the operators, then I quantize which is neccessary and then I do the compiler passes for custom codegens. It is a very simple convolution for now without any padding, so the output width and height are always smaller than the input.

Fusing the operators to my custom conv2d_bias works fine. I made a module only containing my custom operator. I input a (1, 1, 6, 6) data tensor, a (1, 1, 3, 3) kernel tensor and a (1) bias tensor. The output should be (1, 1, 4, 4) which my relation function correctly outputs. However the type solver says it cannot match tensors (1, 1, 6, 6) and (1, 1, 4, 4). I have no idea why the type solver wants to match these two tensors.