Why does ansor not support the tensor core?

#1

Please, why does Ansor not support automatic tuning of the tensor core? I think there should be no problem in the design, but do we need to change more in the implementation?

#2

It’s hard to implement in Ansor because it’s difficult to do pattern marching and tensorization in TE. I would suggest checking meta schedule, which supports tensor core. https://github.com/apache/tvm/blob/main/tests/python/integration/test_meta_schedule_auto_tensorize.py#L351

4 Likes
#3

Thanks for your reply! I will try meta schedule.

#4

I have another question. Do you have any papers or documents on meta schedule for reference? I want to know about the search strategy of meta schedule. Thanks!

#5

Meta-schedule RFC

Meta-schedule Paper

TensorIR/Auto-Tensorize Paper

1 Like
#7

Thanks for your reply! And one last question, what are the limitations of TE compared with TensorIR in terms of expression ability? Will TE be completely abandoned in the future? Thanks!

#8

TE cannot express scatter operator.

In the current compilation stack, TE is fully compatible with TensorIR, and you can use create_prim_func to convert Tensor Expression to TensorIR functions.

TE will be used as a higher-level programming interface rather than being abandoned (it’s an option for users to use TE, users can also bypass TE and use TensorIR to write operators directly).

1 Like
#9

Thanks for all your patient replies!

#10

Out of curiosity, can TE and Ansor realize auto-tensorization if they insist on doing so?

#11

For simple case like GEMM it’s not entirely impossible but would need significant work. For conv, it is hard to implement transformations like im2col automatically in TE schedule. TensorIR is more flexible and we were able to introduce some new schedule primitives for auto tensorization usage.

2 Likes
#12

OK, thanks! By the way, how does TensorIR support dynamic shape now?

#13

TensorIR does not fully support dynamic shape now. It’s not only the problem of TensorIR but also the limitation of the TVM stack. We will start our work for dynamic shape in the near future. :slight_smile:

1 Like
#14

Thanks for your reply! Look forward to your future work!

#15

By the way, when will the version that fully supports Tensor IR be released?

#16

Sorry, I cannot give a specific timeline now. Dyn shape support is a reasearch project, which may face many challenge of uncertainty.

#17

OK. When will 0.10.0 supporting Meta schdule be released?

#18

Meta-schedule is fully functional on mainline

#19

Ok, thanks again, I will try it!