[Tensor Expression] Schedule and Correctness

Questions
1

Does Tensor Expression guarantee that whatever schedules applied (if they can be applied) will not change the output of the algorithm - i.e. the generated code is always correct?

More specifically, given a bug-free algorithm and a set of schedule primitives, there could be three kinds of outcomes.

  1. The schedules have no conflict and a correct code is generated.
  2. The schedules have some conflict and errors are reported at code-gen time.
  3. The schedules have some conflict, but no errors are reported at code-gen time, and an incorrect code is generated.

If 3) possible in Tensor Expression (sans implementation bugs)?

Thanks.

Yuan

2
  1. is possible. I have experienced but cannot find an example at the moment though. maybe vectorize a reduced axis? not sure whether tvm will report error in this case.
3

I also have seen 3. In my case, it happens when I declare the scope of a cache_write to ‘local’.

I wonder whether

a) Tensor Expression’s design allows 3 to happen, or
b) Tensor Expression’s design disallows 3, but it occurs due to an implementation bug ?

4

It is most likely a), but we cannot know for sure without seeing the exact schedule. There are many correctness checks that we do not yet have.

5

Thanks. It seems allowing 3 (i.e. silently generate incorrect code) would make TE a bit tricky to use.

If it is just because of lack of implementation right now, we can improve it over time. Are you aware of any case that is impossible to check at compile time?

6

Could you comment which ones we do not have? (and added bonus to where in the codebase I can find those that we do have)
Thanks

7

This is likely to be very tedious, and the only real suggestion I can give is to try and modify some of the more intricate schedule examples (e.g., the conv2d CUDA tutorial) to see what errors naturally happen. It may be difficult to separate some false-positives from true errors, and likewise some errors may not be deterministic (e.g., violating shared memory semantics) due to run time variability.