Add some new tensorflow ops

Development pre-RFC
#1

Motivation

In order to enrich the support of TF operator and the need of practical work, we add 15 operators to TVM. as follows: * Expm1, Rint, Softsign, Cumprod, Cumsum * SegmentMax, SegmentMin, SegmentMean, SegmentPord, SegmentSum * UnsortedSegmentMax, UnsortedSegmentMin, UnsortedSegmentMean * UnsortedSegmentPord, UnsortedSegmentSum

Implementation Details

  1. Expm1 https://tensorflow.google.cn/api_docs/python/tf/math/expm1 image
  2. Softsign https://tensorflow.google.cn/api_docs/python/tf/nn/softsign image
  3. Rint https://tensorflow.google.cn/api_docs/python/tf/math/rint
  4. Cumprod https://tensorflow.google.cn/api_docs/python/tf/math/cumprod
  5. Cumsum https://tensorflow.google.cn/api_docs/python/tf/math/cumsum
  6. SegmentMax https://tensorflow.google.cn/api_docs/python/tf/math/segment_max
  7. SegmentMin https://tensorflow.google.cn/api_docs/python/tf/math/segment_min
  8. SegmentMean https://tensorflow.google.cn/api_docs/python/tf/math/segment_mean
  9. SegmentPord https://tensorflow.google.cn/api_docs/python/tf/math/segment_prod
  10. SegmentSum https://tensorflow.google.cn/api_docs/python/tf/math/segment_sum
  11. UnsortedSegmentMax https://tensorflow.google.cn/api_docs/python/tf/math/unsorted_segment_max
  12. UnsortedSegmentMin https://tensorflow.google.cn/api_docs/python/tf/math/unsorted_segment_min
  13. UnsortedSegmentMean https://tensorflow.google.cn/api_docs/python/tf/math/unsorted_segment_mean
  14. UnsortedSegmentProd https://tensorflow.google.cn/api_docs/python/tf/math/unsorted_segment_prod
  15. UnsortedSegmentSum https://tensorflow.google.cn/api_docs/python/tf/math/unsorted_segment_sum

In TF frontend, exmp1 composed by exp and subtraction. Softsign composed by abs and divide. Rint replaced by round. We provide a new implementation for Cumsum, Cumprod, SegmentMax, SegmentMin, SegmentMean, SegmentProd, and SegmentSum operators. For the UnsortedSegment operators, they share a set of implementation with Segment operators. The conversion is completed in TF frontend.

@Huyuwei @hlu1 @kazum @siju-samuel @FrozenGene

3 Likes
#2

nice job! If it works out, it will be really helpful!

#3

I think we could just send pr directly. Of course, we could make them be several prs, not one big pr.

#4

I’m curious, how did you implement cumsum? I also wanted cumsum op a while back, but for me it was not clear how it can be implemented efficiently in TVM.

#5

I’ll make them down into several prs, than send them directly. Thank you.

#6

We have tried using te.scan and te.compute to implement cumsum. But it seems that there is no way to use the same formula to adapt to all situations. Finally, we implement it with te.extern.

#7

Thanks, yeah I also remember te.scan only supports scanning along the first axis. I think te.extern is good as a first step.