I would like to comment by point:
- Extract one element from tensor into scalar should be already supported, as we can add getitem operator that gives you back a 0-rank tensor.
- The relay type into value expression need some runtime support and should not be too hard.
- Use of expression in the attribute is something that can cause quite a lot of dynamism. While relay already supports tvm::Expr in the attributes, there needs to be additional runtime support. However, we might be able to get around it in many cases by rewriting, since the dimension is usually known.
The current design trade-off we make is to best support computation where the number of dimensions of the tensor are known, but we do not necessarily known the specific shape value, which should be the common case.
Shape Function Generation
In particular, we would need to add the feature of type function generation to the relay. Which generates a PackedFunc that carries out the shape inference when values are not known. This can be done by passing generic shape var [x, y, z] and get the output shape and dtype relation. Note that not all the TypeRelation are written to support this way(because some rely on constant), and need to be modified to work for the general cases. Similarly, when some of the operators allow attributes to contain tvm::Expr, the TypeRelation provided so far may not handle them, and need to be upgraded (or create a new Op attribute to do so).