Using the Graph Executor together with CMSIS-NN I get runtime errors like
no such function in module: tvmgen_default_cmsis_nn_main_0
In the generated code the above function uses the CMSIS-NN kernel but it is missing in the function registry, so it cannot be triggered by the RPC calls.
Are Graph Executor and CMSIS-NN supposed to be compatible (and it is a bug) or are they mutual exclusive?
Hi @Manuel,
CMSIS-NN was designed to work with the Ahead of Time executor, so you may run into some issues using the Graph Executor.
Thanks for the answer. Does it mean that conceptually there is no reason why Graph Executor would not work with CMSIS-NN. It is just not implemented/verified in microTVM. Correct?
Wouldn’t it be a good feature then? The Graph Executor is used during auto-tuning to improve the performance. Why should microTVM sacrifice this advantage by using a self-generated kernel that does not make as much use of the SIMD capabilities of the target hardware (Cortex-M4) as the CMSIS-NN kernel does?
In the current implementation, CMSIS-NN support relies on the target hook system. This system supported by the Ahead of Time Executor, while it’s not been used with the Graph Executor. I don’t see any reason CMSIS-NN wouldn’t work with the Graph Executor, it’s just a case of enablement and adding the plumbing.
Currently we rely on CMSIS-NN kernels being greedily partitioned where they are supported. Tuning might be able to make better decisions about what kernel to use, although I’m not aware of a compilation flow that exists today capable of doing that.
Hi @Manuel , @lhutton1 i am running into the same issue , tvm_func_registry has only two functions: static const TVMFuncRegistry tvm_func_registry = { "\002\000tvmgen_default_fused_reshape\000__tvm_main\000", _tvm_func_array, };
and my case run without calling any function , had you find a way to make it work? is there a plan to enable that?
Hi @ebraheema, I didn’t get it working. For the moment, I am using the AoT Executor.