[Question] Does the current vision auto_scheduler support search using wmma?

Questions
#1

above question, any none knows?

Does the current autoscheduler support generating tensorcore code for NVIDIA GPUs?
#2

It’s not supported. However, meta schedule does. We recently upstreamed auto tensorization and now available in the main branch. Here’s an example tuning with meta schedule tune_gemm.py · GitHub

1 Like
#3

Thank you for your reply! I read the paper TensorIR: An Abstraction for Automatic Tensorized Program Optimization. It says TensorIR run bert-large model can be as fast as TensorRT. Does it use meta_schedule for search better op implementation?

When I use auto_schedule search the model exported from tensorflow(bert-base batch 8, seq 384), it can achive about 100 fps. FasterTransformers can be 1100fps. Should I use meta_schedule for search？

@vinx13

#4

Yes it uses meta schedule, which uses tensorization (CUDA fp16 tensor core) and software pipelining for optimization.

1 Like
#7

Do the order of rules have any influence on the meta schedule? I think the order of rules cannot be changed in Ansor before. @vinx13

#8

The order matters. It is applied to the workload in the specified order.

#10

Ok, If I want to see the changes after each rule is applied, how can do it?

#11

You can add some logging here https://github.com/apache/tvm/blob/main/src/meta_schedule/space_generator/post_order_apply.cc#L173. You can also run each separately (see test case for example https://github.com/apache/tvm/blob/main/tests/python/unittest/test_meta_schedule_schedule_rule_mlt.py)

#12

Ok, thanks for your reply!