Loop partitioning and Tensorization - Work on different IR levels

[aca88]

Hey.

@baowenlei1

do you mean after you set the loop partition with build_config, the tensorization works without throw previous error

No sorry that was not what I meant.

I gave up on the idea of using the tensorize scheduling operation because ir_pass.LoopPartition works at a different level (i.e. AST level) and I didnt feel like writing an intrin_func (needed for tensorize) template that would work for all sizes of my tiles (i.e. for the tail regions were the tiles are smaller than normal). So I added an ir_pass in the last stage which (to my understanding) does something equivalent to what you would expect tensorize to do and at least for my purpose it works.

The way I did it is very similar (if not identical to this VTA construct), which I think is very similar to

@xqdan

The way we solve this issue is,

1. use a new tensorize api we name it eimt_insn to replace tensorzie in schedule, which just put a pragma on the head of loops.
2. peel the loops into body and tail, and loops in both of them have the pragma.
3. emit insn pass will call callback you’ve defined for the pragma, this pass will give the callback loop IR, in which you will have all the information of tensorization.
4. in predefined callbacks, we check and transform the input ir to the instructions we want.

1. Instead of a new api, use pragmas
2. I let the ir_pass.LoopPartition peel the loops for me
3. Same as above
4. Same as above

[baowenlei1]

Thanks for the reply. I feel that would be too much change for my case. I am working on my walkaround, and I think

writing an intrin_func (needed for tensorize ) template that would work for all sizes of my tiles

it is possible do achieve that, depends on how to write the tensorize kernel.

[WanweiCAI]

Hi, @baowenlei1，have you resolved the “likely(xxx)” tensorization problem? or the “pragmas” is the only way to do tiling correctly? Thanks！

[baowenlei1]

Have not check this site for a while. You can check the forked branch here https://github.com/microsoft/onnxruntime-tvm

[WanweiCAI]

Thanks for your reply!

[ilya]

Hi! I’ve also met the problem with “likely(…)” statements in tensorization. After enabling loop partitioning with “partition_const_loop” in config, some of them disappeared, but not all. Are there any way to completely remove “likely(…)” statements before tensorization, without writing your own IR pass?