[AOT] Problem in microTVM (uTVM) host driven application

Troubleshooting
#1

I am building a model using the AoT executor and the CRT runtime, trying to run the model in a host driven manner, because I want to compare it with the Graph executor (I noticed @alanmacd added this to the CRT runtime, so perhaps he can give me a hint).

What I am doing is the following:

...
RUNTIME = tvm.relay.backend.Runtime("crt", {"system-lib": True})
TARGET = tvm.target.target.Target({"kind": "c", "link-params": True})
if executor == "aot":
    EXECUTOR = tvm.relay.backend.Executor("aot",options={'interface-api': 'c','unpacked-api': 1})
elif executor == "graph":
    EXECUTOR = tvm.relay.backend.Executor("graph")
with tvm.transform.PassContext(opt_level=3, disabled_pass=["AlterOpLayout"]):
    module = relay.build(mod, executor=EXECUTOR ,runtime=RUNTIME, target=TARGET, params=params)
...
with tvm.micro.Session(generated_project.transport()) as session:
    if executor == "graph":
        module_executor = tvm.micro.create_local_graph_executor(
            module.get_graph_json(), session.get_system_lib(), session.device
        )
    else:
        module_executor = tvm.runtime.executor.aot_executor.AotModule(session.create_aot_executor())
    module_executor.set_input("predict_images:0",np.expand_dims(img,0))
    module_executor.run()
...

When using the Graph executor, the model works fine. But when selecting the AoT executor, I get the following error:

[14:23:48] .../tvm/src/runtime/micro/micro_session.cc:368: remote: .../generated-project/src/standalone_crt/src/runtime/crt/aot_executor/aot_executor.c:81: Check failed: rv >= 0: cannot find 'predict_images:0' among input.
[14:23:48] .../tvm/src/runtime/micro/micro_session.cc:368: remote: TVMPlatformAbort: 0x00000500

I see this output when running the “with” clause:

...
[14:23:48] .../tvm/src/runtime/micro/micro_session.cc:368: remote: DumpMetadata:
[14:23:48] .../tvm/src/runtime/micro/micro_session.cc:368: remote:     mod_name=tvmgen_default
[14:23:48] .../tvm/src/runtime/micro/micro_session.cc:368: remote:     version=1
[14:23:48] .../tvm/src/runtime/micro/micro_session.cc:368: remote:     num_inputs=0
[14:23:48] .../tvm/src/runtime/micro/micro_session.cc:368: remote:     num_outputs=0
[14:23:48] .../tvm/src/runtime/micro/micro_session.cc:368: remote:     num_workspace_pools=0
[14:23:48] .../tvm/src/runtime/micro/micro_session.cc:368: remote:     num_constant_pools=0
...

So the question is:

  1. Am I using it correctly?
  2. Does the AoT executor change the name of the input of the model?
  3. Is there any step I am missing?
#2

@fPecc I think when using the AOTExecutor via RPC you need to remove {'interface-api': 'c','unpacked-api': 1} from the Executor() definition. Could you give that a shot? Otherwise I don’t think the metadata will properly print. We should add a better assert around that, filed https://github.com/apache/tvm/issues/12060

#3

Hi @areusch thanks for the answer!

Sadly, it did not work. Not matter which options I pass to the AoTExecutor, I always become an empty metadata.

#4

@fPecc, not sure if this is still relevant. I also ran into the ‘cannot find ‘predict_images:0’ among input’ error with the AoT Executor while the Graph Executor is fine. I realized that the AoT Executor replaces the colon in the input name by an underscore. So, try ‘predict_images_0’. The Graph Executor does not seem to have this property.

@areusch why are Graph Executor and AoT Executor treating the input name differently? It is not very intuitive. Is it documented somewhere?

#5

I found the offending line in the source code: aot_executor_codegen.cc does

std::string input_name = SanitizeName(input->name_hint());

But I don’t know if this is simply missing in the Graph Executor or if it superfluous in the AoT Executor.

#6

I filed a ticket in