Qualcomm SNPE and TVM

Questions
#1

Qualcomm SNPE is effectively a ML hardware runtime scheduler that provides a user space API that enables many applications to share compute resources: Snapdragon Neural Processing Engine SDK: Features Overview. Not all OP codes are supported, and the SDK provides a hybrid model for OP codes not supported; CPU callback on specific OP codes which get implemented in Apps Proc native code. It’s complicated for the standard ML engineers who know little about hardware (most), but it is mandatory for a heavily loaded system.

I like the vendor neutral approach of TVM, and it could make sense to replace SNPE. I’m trying to understand the gaps.

Questions:

  1. Does TVM implement construct of a hardware resource scheduler that handles context switching of hardware resource? If it exists is there a working example?

  2. How does power management tie into TVM, if at all?

Thanks, Joel

1 Like