Pass Instrument Framework Proposal

Development RFC
4

Thanks for the PR @zackcquic.

  1. Why would you like to keep runtime profiling and pass profiling separate? The benefit I see is that a lot of the code is similar. We could avoid a lot of code duplication. On the other hand runtime profiling has does have a lot of code around handling timing on different devices, which would not be used for pass profiling.

  2. I think we need consistent naming around profiling/instrumentation. I’d prefer names like tvm.pass.profiling and tvm.runtime.profiling.

  3. Reading through your PR, there seems to be two interfaces for adding a new PassInstrument. Either you can subclass PassInstrument or you can register callbacks. I’d like to see a single API. Subclassing PassInstrument is more in line with how we normally handle extensibility in the codebase. It also a better interface if the instrument needs to maintain some internal state.

  4. What instruments do you see being useful in the future? Time and memory usage are the only ones that come to mind for me.

  5. What are parts of the code do you imagine we will instrument/profile in the future? (It might be useful to talk about adding a more general to do instrumentation in the codebase, but maybe that is not appropriate for this discussion).

@areusch May have some feedback too.