I was wondering if there is strong need to in-place mutation. Introducing in-place operator is a bit troublesome and the gain is usually little, because most of those operations can be inlined. For example, in the conv2d+relu usecase, we can fuse relu into conv2d.