Hi, I am trying to benchmark a few models using the Cortex-M55 MCU in combination with the Ethos-U55 NPU. The models are TFlite files which I am trying to convert to C inference code using microTVM. The conversion command is taken directly from the Bare-Metal tutorial:
tvmc compile \
--target=ethos-u,cmsis-nn,c \
--target-ethos-u-accelerator_config=ethos-u55-128 \
--target-c-mcpu=cortex-m55 \
--runtime=crt \
--executor=aot \
--executor-aot-interface-api=c \
--executor-aot-unpacked-api=1 \
--pass-config tir.usmp.enable=1 \
--pass-config tir.usmp.algorithm=hill_climb \
--pass-config tir.disable_storage_rewrite=1 \
--pass-config tir.disable_vectorize=1 \
--output-format=mlf \
--verbose \
model.tflite
This works when using a regular CNN (MobileNetV2) that can be offloaded to the U55 entirely. It also works for a Vision Transformer model (MobileViT XXS) when disabling the U55 and running the entire model on the M55. However, the combination U55+MobileViT yields an internal error:
InternalError: Check failed: (checked_type_.defined()) is false: internal error: the type checker has not populated the checked_type field for CallNode(...
Error Trace
- set -e
- tvmc compile --target=ethos-u,cmsis-nn,c --target-ethos-u-accelerator_config=ethos-u55-128 --target-c-mcpu=cortex-m55 --runtime=crt --executor=aot --executor-aot-interface-api=c --executor-aot-unpacked-api=1 --pass-config tir.usmp.enable=1 --pass-config tir.usmp.algorithm=hill_climb --pass-config tir.disable_storage_rewrite=1 --pass-config tir.disable_vectorize=1 --output-format=mlf --verbose mobilevit_integer_quant_30.tflite Traceback (most recent call last): File “/home/elias/dbsparse/.venv/bin/tvmc”, line 8, in sys.exit(main()) File “/home/elias/dbsparse/.venv/lib/python3.9/site-packages/tvm/driver/tvmc/main.py”, line 118, in main sys.exit(_main(sys.argv[1:])) File “/home/elias/dbsparse/.venv/lib/python3.9/site-packages/tvm/driver/tvmc/main.py”, line 106, in main return args.func(args) File “/home/elias/dbsparse/.venv/lib/python3.9/site-packages/tvm/driver/tvmc/compiler.py”, line 201, in drive_compile compile_model( File “/home/elias/dbsparse/.venv/lib/python3.9/site-packages/tvm/driver/tvmc/compiler.py”, line 382, in compile_model mod = partition_function(mod, params, mod_name=mod_name, **opts) File “/home/elias/dbsparse/.venv/lib/python3.9/site-packages/tvm/relay/op/contrib/ethosu.py”, line 60, in wrapper return func(args, kwargs) File “/home/elias/dbsparse/.venv/lib/python3.9/site-packages/tvm/relay/op/contrib/ethosu.py”, line 2402, in partition_for_ethosu mod = preprocess.preprocess_ext_io()(mod) File “/home/elias/dbsparse/.venv/lib/python3.9/site-packages/tvm/ir/transform.py”, line 160, in call return _ffi_transform_api.RunPass(self, mod) File “tvm/_ffi/_cython/./packed_func.pxi”, line 332, in tvm._ffi._cy3.core.PackedFuncBase.call File “tvm/_ffi/_cython/./packed_func.pxi”, line 263, in tvm._ffi._cy3.core.FuncCall File “tvm/_ffi/_cython/./packed_func.pxi”, line 252, in tvm._ffi._cy3.core.FuncCall3 File “tvm/_ffi/_cython/./base.pxi”, line 182, in tvm._ffi._cy3.core.CHECK_CALL File "/home/elias/dbsparse/.venv/lib/python3.9/site-packages/tvm/ffi/base.py", line 481, in raise_last_ffi_error raise py_err tvm.error.InternalError: Traceback (most recent call last): 22: ZN3tvm7runtime13PackedFuncObj9ExtractorINS0_16PackedFuncSubObjIZNS0_15TypedPackedFuncIFNS_8IRModuleENS_9transform4PassES5_EE17AssignTypedLambdaINS6_UlS7_S5_E5_EEEvT_SsEUlRKNS0_7TVMArgsEPNS0_11TVMRetValueEE_EEE4CallEPKS1_SD_SH 21: tvm::transform::Pass::operator()(tvm::IRModule) const 20: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const 19: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const 18: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const 17: tvm::transform::ModulePassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const 16: ZN3tvm7runtime13PackedFuncObj9ExtractorINS0_16PackedFuncSubObjIZNS0_15TypedPackedFuncIFNS_8IRModuleES5_NS_9transform11PassContextEEE17AssignTypedLambdaIZNS_5relay9transform24PreprocessExternalFuncIOEvEUlS5_S7_E_EEvT_EUlRKNS0_7TVMArgsEPNS0_11TVMRetValueEE_EEE4CallEPKS1_SF_SJ 15: tvm::relay::contrib::ethosu::PreprocessExternalFuncIO(tvm::IRModule const&) 14: tvm::relay::PostOrderRewrite(tvm::RelayExpr const&, tvm::relay::ExprRewriter) 13: tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&) 12: void tvm::relay::ExpandDataflow<tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1}, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#2}, tvm::relay::ExpandDataflow<{lambda(tvm::RelayExpr const&)#1}, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1}>(tvm::RelayExpr, {lambda(tvm::RelayExpr const&)#1}, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1})::{lambda(tvm::RelayExpr const&)#1}>(tvm::RelayExpr, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#2}, tvm::relay::ExpandDataflow, tvm::relay::ExpandDataflow<{lambda(tvm::RelayExpr const&)#1}, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1}>(tvm::RelayExpr, {lambda(tvm::RelayExpr const&)#1}, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1})::{lambda(tvm::RelayExpr const&)#1}) [clone .isra.0] 11: tvm::relay::MixedModeMutator::VisitLeaf(tvm::RelayExpr const&) 10: tvm::relay::PostOrderRewriter::DispatchVisitExpr(tvm::RelayExpr const&) 9: ZZN3tvm5relay11ExprFuncto 8: tvm::relay::ExprMutator::VisitExpr(tvm::relay::FunctionNode const) 7: tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&) 6: void tvm::relay::ExpandDataflow<tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1}, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#2}, tvm::relay::ExpandDataflow<{lambda(tvm::RelayExpr const&)#1}, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1}>(tvm::RelayExpr, {lambda(tvm::RelayExpr const&)#1}, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1})::{lambda(tvm::RelayExpr const&)#1}>(tvm::RelayExpr, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#2}, tvm::relay::ExpandDataflow, tvm::relay::ExpandDataflow<{lambda(tvm::RelayExpr const&)#1}, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1}>(tvm::RelayExpr, {lambda(tvm::RelayExpr const&)#1}, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::{lambda(tvm::RelayExpr const&)#1})::{lambda(tvm::RelayExpr const&)#1}) [clone .isra.0] 5: tvm::relay::MixedModeMutator::VisitLeaf(tvm::RelayExpr const&) 4: tvm::relay::PostOrderRewriter::DispatchVisitExpr(tvm::RelayExpr const&) 3: tvm::relay::ExprRewriter::Rewrite(tvm::RelayExpr const&, tvm::RelayExpr const&) 2: ZZN3tvm5relay12ExprRewrit 1: tvm::relay::contrib::ethosu::ExternalFuncIOHandler::Rewrite(tvm::relay::CallNode const, tvm::RelayExpr const&) 0: tvm::RelayExprNode::checked_type() const [127 127 127 … 127 127 127]] [127 127 127 … 127 127 127] [127 127 127 … 127 127 127] … [127 127 127 … 127 127 127] [127 127 127 … 127 127 127] File “/workspace/tvm/include/tvm/ir/expr.h”, line 741 InternalError: Check failed: (checked_type.defined()) is false: internal error: the type checker has not populated the checked_type field for CallNode(I.GlobalVar(“tvmgen_default_ethos_u_main_11”),
… (truncated because of character limit)
Python version: 3.9.5 vela version: 3.12.0 tvmc version: 0.14.dev273 (testing with 0.13 yields the same error)
I suspect that this is related to certain operations (Reshape, Transpose, …) not being supported on the U55 and the required switching between M55/U55 causing issues at the interface between vela and microTVM. Is there any way of fixing this?