We discussed this at the TVM Community Meeting this morning (follow the link for the recording). Here are notes (thanks to @mbs-octoml for taking these–I’m just reposting them):
Tristan: How does typing work with dynamic shapes?
Yuchen: Types and shapes are separate. Type carries only rank and dtype. Shape is deferred to the expression level, eg Array<tir.Var>.
TQ: Avoids dependent typing since don’t need to decide shapes at type checking time. Constant propagation and affine expression simplification still applied.
Andrew: One thing we could do in Relay is express program without defining implementation of primitive ops. Can we still do this in Relax? Seems we want to eagerly unfold ops to PrimFuncs.
Yuchen: More flexible in Relax, since can write your own call_packed etc. In future will defer expansion.
Chris: Talked about progressive lowering. Can we inspect Relax IR at every step?
Yuchen: Every step is IRModule->IRModule, can print or even build at any point.
Andrew: How do you see this landing in TVM main? Eg are the VMs the same.
Yuchen: Relax has own VM (eg has its own shape heap & instructions). Reduced num instructions, deferred most to call_packed with packed funcs in the runtime. This Q robustifying key components w/ community. Q3 upstream as a separate codepath in own code dir.
Andrew: TE needs changes for symbolic shapes?
Yuchen: Relax ‘works backwards’ from TE’s existing symbolic shape machinery so no changes.
Andrew: AOT and Relax VM? Perhaps fewer instructions makes easier?
Yuchen: Currently not working on AOT executor.
Chris: Would be good to codegen those extension functions.
Michael: On future work: Usability for ONNX as input? Plans for getting to front-ends?
Yuchen: Working on matching perf w/ Relax. Needs passes like FuseOps to match static perf. Then move on to dynamic. Pytorch importer starting in Q3. If more contrib then can build ONNX importer, not started.
Manupa: Can relay and relax co-exist in a single IRModule?
Yuchen: No, different ASTs