[RFC] Differentiable tensor expression (Create and verify backward op automatically)

[yzhliu]

@wrongtest

If I have some common neural network structure such as resnet50 at hand, can I just use autodiff to get backward computation graph?

graph-wise I think you can refer to relay.transform.gradient and as you lower the differentiated graph, you may leverage the tensor-level autodiff (te.gradient). Though tensor gradients now are mostly manually written.

@wrongtest

Is there some description about common ops which can be coveraged by autodiff?

You may refer to test cases

@wrongtest

Can te.scan() be supported?

currently it is not supported.