Add dynamic shape for pytorch expand converter op

Questions
1

I met following error when I try to feed input tensor with dynamic batch size: (bert model)

File “/yezhouhai/xtcl-master/baidu/xpu/xmir/python/tvm/relay/frontend/common.py”, line 529, in infer_value ), “All inputs to infer must be available in params.”

This is because in function (pytorch.py) expand: sizes[i] = int(_infer_value(sizes[i], {}).asnumpy()) This sizes[i] will be feeded to input of op.repeat (2nd parameter) which only accept int parameter.

But in static case, the sizes is constructed by following: sizes: [Constant(1), Constant(8)] In dynamic case, the sizes is constructed by following: sizes: [CallNode(Op(take), [CallNode(Op(shape_of), [Var(input_ids, ty=TensorType([?, 8], float32))], relay.attrs.ShapeOfAttrs(0x7f169cefa318), []), Constant(0)], relay.attrs.TakeAttrs(0x7f169cefabc8), []), Constant(8)]

So in dynamic case, the sizes[0] is a relay.Expr.Call and it can’t be inferred as int.

Two possible solution I can think of:

  1. I try to reuse onnx’s expand version. But find it’s input shape is relay.Expr, but in our case the shape is a list type. I don’t know how to convert a list to shape.
  2. Write a dyn.repeat op? So that it can accpet relay.Expr as its parameter?

Any suggestions? Thanks!

2

I think you can use op.concatenate(...) to turn a list of Expr into a single Expr representing a dynamic shape. The second solution also sounds reasonable.