Incremental Type Propagation

t-vi · November 11, 2020, 8:39pm

I made this, in the end I stayed in the PyTorch frontend and solved it locally given that inplace things don’t look like TVM that much.

github.com/apache/tvm

PyTorch frontend: make type inference incremental

main ← t-vi:incremental_type_inference

opened 01:45PM - 11 Nov 20 UTC

t-vi

+214 -131

Currently, the PyTorch frontend will use the vanilla TVM type inference pass to …get the types. Combined with the incremental nature of translating the graph, this makes for quadratic (in the graph size) runtime. This patch runs type inference on a subgraph (starting from the things that has types) instead. It is a bit hacky though because it essentially tries to implement in-place modification where TVM does not foresee it. For converting Huggingface BERT, this patch gives a ~10x increase in the speed of conversion (from 31 seconds to just below 3), so it is solving a very real and quite bad problem.