Hi, @tqchen Is there any plan on the “Fully Featured Relay AOT” Compiler?
We have a dynamic-shaped model, which is memory-intensive (the execution time of a single operator is short), the vm’s approach is hard to meet our performance requirements, the overhead is too large compared to the operators execution time.
So, we want a solution to minimize the overhead introduced by the extra code related to shape calculation and memory management.
AOT solution sounds like a good choice, since we can inline the computation related to shape calculation and eliminate the overhead related to interact with the VM.