Using the Graph Executor together with CMSIS-NN I get runtime errors like
no such function in module: tvmgen_default_cmsis_nn_main_0
In the generated code the above function uses the CMSIS-NN kernel but it is missing in the function registry, so it cannot be triggered by the RPC calls.
Are Graph Executor and CMSIS-NN supposed to be compatible (and it is a bug) or are they mutual exclusive?
Thanks for the answer. Does it mean that conceptually there is no reason why Graph Executor would not work with CMSIS-NN. It is just not implemented/verified in microTVM. Correct?
Wouldn’t it be a good feature then? The Graph Executor is used during auto-tuning to improve the performance. Why should microTVM sacrifice this advantage by using a self-generated kernel that does not make as much use of the SIMD capabilities of the target hardware (Cortex-M4) as the CMSIS-NN kernel does?
In the current implementation, CMSIS-NN support relies on the target hook system. This system supported by the Ahead of Time Executor, while it’s not been used with the Graph Executor. I don’t see any reason CMSIS-NN wouldn’t work with the Graph Executor, it’s just a case of enablement and adding the plumbing.
Currently we rely on CMSIS-NN kernels being greedily partitioned where they are supported. Tuning might be able to make better decisions about what kernel to use, although I’m not aware of a compilation flow that exists today capable of doing that.
Hi @Manuel , @lhutton1
i am running into the same issue , tvm_func_registry has only two functions:
static const TVMFuncRegistry tvm_func_registry = {
"\002\000tvmgen_default_fused_reshape\000__tvm_main\000", _tvm_func_array,
};
and my case run without calling any function , had you find a way to make it work?
is there a plan to enable that?