[Bundle] demo_static compile errors

Hello,

While compiling the bundle_deploy demo_static as given in the readme, I encountered the following errors:

demo_static.c: In function ‘main’:
demo_static.c:52:3: warning: ignoring return value of ‘fread’, declared with attribute warn_unused_result [-Wunused-result]
   (void)fread(input_storage, 3 * 224 * 224, 4, fp);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
build/model_c.o: In function `fused_nn_softmax':
TVMMod:(.text+0x2667): undefined reference to `expf'
TVMMod:(.text+0x2678): undefined reference to `expf'
TVMMod:(.text+0x268a): undefined reference to `expf'
TVMMod:(.text+0x269d): undefined reference to `expf'
build/backtrace.o: In function `tvm_platform_abort_backtrace':
/home/tvm/apps/bundle_deploy/backtrace.c:43: undefined reference to `dladdr'
collect2: error: ld returned 1 exit status
Makefile:94: recipe for target 'build/demo_static' failed
make: *** [build/demo_static] Error 1

Especially concerning are the expf not being defined since I would assume that they are available from math.h

cc: @areusch since you seem have the last commit in a couple of the files in that folder

@cron thanks for this. looks like we had missed a few things from the experimental branch.

I’ve opened PR 6630 which should address this. Could you verify it works? Feel free to comment on the PR.

Hey,

It doesnt seem to fix any of the problems I listed (i.e. the error messages did not change).

I think the expf problems didnt get fixed because the build_model.py calls relay.build with llvm as target. Your inclusion of math.h is for the C source code generator.

ah that’s right. looks like now we just have a problem in apps/bundle_deploy/Makefile (it needs to link with -lm). I’ll submit a separate PR to fix this, since one from my original reply has passed regression.

Wow that’s an easy fix… I guess that’s a good thing.

:slight_smile:

Would you mind posting here exactly where to add the -lm ? I tried a couple of ways and still get same error

give it a shot with this PR: https://github.com/apache/incubator-tvm/pull/6636

1 Like

Yes worked. Thanks a lot.

I still get the undefined reference to ‘dladdr’ but I think the reason might be something in my environment

sorry I missed this one–dladdr requirement can be disabled if you specify make ENABLE_TVM_PLATFORM_ABORT_BACKTRACE=0. this was just an incidental thing we put in to the demo to try and catch a flaky regression problem we couldn’t reproduce locally. we haven’t seen it trigger yet, I believe.

Hey,

Thanks for answering. I was able to deal with some of the dladdr problems by commenting out part of the code (which I guess had the same effect as the solution you proposed).

I have been having problems trying to do the following:

I want to be able to compile the bundle_deploy example for arm, with no OS. That also includes without Zephyr.

  • I know I have to change here to relay.build('llvm -mtriple=arm-none-eabi') so that the right llvm target machine is used. Anything else I need to set here?

  • Could you maybe elaborate on what --system-lib does?

  • I also know that the Makefile requires some changes since it does not seem to account for 'arm-none-eabi’gcc prefix. Weirdly enough the Makefile for the crt libraries does accept this prefix. Am I missing something here?

  • Could you help me with defining the flags required for compiling in this case? (I see that some flags are defined in the Makefile, but I cant imagine that those are enough for arm embedded bare metal target)

  • Do you know of any reason why any of the modules required for the bundle or libraries are not bare metal compatible? (I tried to set up a compilation pipeline but I was getting newlib errors and it felt it had something to do with exception handling but I am not sure. I did however always try to compile with backtracing on).

Thanks for the help :slight_smile:

hi @cron,

glad you were able to solve the dladdr problems.

there are a couple of ways to run TVM operators in a bare-metal environment. a lot of the recent work has centered around creating a firmware binary that includes compiled TVM operators plus an RPC server which can be commanded to run them over e.g. a UART. this is slightly different from what you’re asking, but let me give you some pointers to that code since the generated operators are compatible (actually, exactly the same at present) between this approach and a fully standalone firmware binary:

  • first i’d point you to the recent test_zephyr.py as a starting point. I know you want to run without Zephyr–the generated code is agnostic to the presence or absence of Zephyr. you will need to provide some replacement platform hooks, but i’ve tried this and it’s not intended to be difficult.

  • another resource i’ll point you to is my recently uploaded virtual machine branch (soon, PR), which makes a couple changes to the transport layer to make things work with real hardware using a UART connection. it also checks in a Vagrant base box + another Vagrantfile, to be used together to bring up a local Virtual Machine with Zephyr + cross compilers for ARM, RISC-V, etc. I’ve tested this same script (test_zephyr.py) against both real hardware and QEMU on this branch.

now as to the question of creating a standalone binary: the main difference is that the TVM Graph Runtime needs to execute on the device. apps/bundle_deploy demonstrates a way to run Graph Runtime in a separate application from TVM; however, it relies on some linux filesystem calls to load data. you could run standalone on bare-metal by porting apps/bundle_deploy to your SoC. here’s roughly what I think you need to do to create a standalone µTVM firmware image that runs your model:

  1. as you mentioned, adjust the target string to 'c --system-lib -mcpu=cortex-mN' (you can follow the examples here). (you can also use llvm in place of c as bundle_deploy does, but this may be less readable).
  2. create a version of apps/bundle_deploy/demo_static.c (note: only demo_static is worth trying to port, no other bundle_deploy targets are portable to bare metal) that loads input_storage using whichever way your platform provides. i’d imagine for a simple bare-metal ARM example, you could use just a flash array in the same way build_graph_c_json is handled. Presumably you could also read this from a sensor.
  3. create the binary firmware image: take the generated .c (if using c target) or .o (if using llvm target) file created by apps/bundle_deploy/build_model.py, link it against your revised demo_static.c and the required runtime libraries in build/standalone_crt, specifically libcommon and libgraph_runtime.

I should also mention that test_zephyr.py is doing something slightly different from apps/bundle_deploy, but I’m giving instructions for something similar to bundle_deploy since that’s what you’re trying to do. specifically, test_zephyr.py is building a firmware image containing a TVM RPC server, then sending it commands to execute operators over a UART. the firmware created with test_zephyr.py cannot run a model without the PC connected.

here are some answers to your questions:

I know I have to change here to relay.build('llvm -mtriple=arm-none-eabi') so that the right llvm target machine is used. Anything else I need to set here?

answered above

Could you maybe elaborate on what --system-lib does?

see docs here

I also know that the Makefile requires some changes since it does not seem to account for 'arm-none-eabi’gcc prefix. Weirdly enough the Makefile for the crt libraries does accept this prefix. Am I missing something here?

yep, so bundle_deploy is not targeted to a cross-compiled nor bare-metal system. it’s meant to be compiled on the host it runs on, and expects linux for basic file operations. the actual model execution part in bundle_deploy's demo_static is portable. this part is made up of CRT libraries and the operator code generated by build_model.py. The Makefile for the CRT libraries is thus a little more flexible and can be retargeted with PREFIX=. Please make sure you’re using build/standalone_crt/Makefile when trying to use that build system–even though it’s the same file, you can’t use the Makefile placed at src/runtime/crt/Makefile because it expects certain paths to be placed in the correct spot (see cmake/modules/StandaloneCrt.cmake).

Could you help me with defining the flags required for compiling in this case? (I see that some flags are defined in the Makefile, but I cant imagine that those are enough for arm embedded bare metal target)

I can’t give too many specifics as I don’t know what SoC you’re compiling for. I’d say SoC-specific bringup will be hit-or-miss here, because it’s fairly specialize to your project–this is why we’ve opted to link against RTOS such as Zephyr in the hopes of producing something more broadly useful. Hopefully you can work off of some example code there to start with.

That said, feel free to ask if you have compilation or runtime problems and I or others will try and help as we can. I’m also particularly interested in talking about any improvements to the overall compilation flow or the CRT libraries that would make µTVM easier to use in everyone’s projects, since there will always be constraints from other projects that developers don’t know about.

Do you know of any reason why any of the modules required for the bundle or libraries are not bare metal compatible? (I tried to set up a compilation pipeline but I was getting newlib errors and it felt it had something to do with exception handling but I am not sure. I did however always try to compile with backtracing on).

No, the generated modules and CRT is bare-metal compatible. apps/bundle_deploy, however, defines 3 possible execution flows:

  • demo_static using CRT
  • demo_dynamic using CRT
  • demo_dynamic using C++ runtime (I.e. parts of libtvm_runtime.so)

if you try to link the code from the latter two, you’ll run into exception problems and problems with dlopen.

hope this helps–feel free to ask more questions and i’ll try and help as I can.

Andrew

3 Likes

Hey Andrew,

Thanks a lot for your answer. There is a lot of information there for me to unpack and it will take me some time.

I appreciate the work done in the direction of uTVM and Zephyr integration, and I think is very valuable work. The thing I want to investigate is if there are any limitations to the C runtime. Therefore the current host+devboard setup wouldnt help me find these weaknesses.

I will be going through your notes and trying to get it to work for my use case. I am sure I will have some hurdles and will post here.

Thanks a lot :slight_smile: