Confused about C/C++ runtimes

Questions
#1

Hi, my understanding is that uTVM uses a pure C runtime (microTVM Design Document — tvm 0.18.dev0 documentation) however when I compile with

python -m tvm.driver.tvmc compile test.onnx --verbose --target='c -keys=cpu' --runtime=crt --runtime-crt-system-lib=1 --executor=aot --executor-aot-interface-api=c --executor-aot-unpacked-api=1 --pass-config tir.disable_vectorize=1 --output-format=mlf --output=model_tvm.tar

I get the error

minrpc_server.h:38:10: fatal error: 'memory' file not found
#include <memory>

This confuses me because this seems to be a c++ header, and I did not think any c++ code would be generated. How can I turn off C++?

The design document also references a “standalone” and “host” mode for uTVM, but it is not obvious to me how to control this.

Any help is greatly appreciated :slight_smile:

Thanks

#2

I think I have worked it out.

It seems c++ code is still exported, but I dont actually need to compile it to get a runnable standalone binary.

I can comment out the rpc libraries (which i think are the “host” mode components?)

  set(CRT_LIBS 
    #microtvm_rpc_server
    #microtvm_rpc_common
    aot_executor_module
    aot_executor
    graph_executor_module
    graph_executor
    common
    memory
  )

and my test model still compiles and runs but now in pure C.

Is there any way to avoid exporting the rpc code? Is this a mistake?

#3

I suppose this makes sense. Internally the python for the tvmc command seems to just copy a load of code from the TVM source tree and bundle it in the tar file. The runtime and stuff doesn’t need to be generated. Only the model code is actually generated (makes sense). So if it is just copying a source containing uTVM host and standalone runtimes from the tvm source tree, it will include the c++.

I suppose I can just compile/use the rpc libraries if I decide to use host mode.

Would be grateful if someone would confirm my theory :slight_smile:

#4

Okay well it turns out I don’t need any of these libraries. I can just provide the C sources

add_executable(test 
	${CMAKE_CURRENT_SOURCE_DIR}/app.c
	${CMAKE_CURRENT_SOURCE_DIR}/platform.c
	${CMAKE_CURRENT_SOURCE_DIR}/out/codegen/host/src/default_lib0.c
	${CMAKE_CURRENT_SOURCE_DIR}/out/codegen/host/src/default_lib1.c
	${CMAKE_CURRENT_SOURCE_DIR}/out/runtime/src/runtime/crt/memory/page_allocator.c

)

and it builds and appears to run an inference.

Could someone please help me understand what the runtime is providing? Is it just for the host mode?