For a number of use cases in TVM, it would be valuable to support in-place operators. One such operator would be strided_set, which currently takes two tensors, a primary tensor and a subtensor, and writes the subtensor into the primary tensor at a particular offset. It doesn’t do this in-place though, so in practice a new third tensor is created.
Were an in-place strided_set to exist, this would allow for some useful patterns. For example, a zero-copy concatenate where the necessary tensors are directly written into the correct part of the larger concat tensor.
Has anyone done any work on in-place operators? It’s not obvious to me what would need to be done to support them in TVM, so I’d be interested to find out if anyone has any ideas.