Hi everyone, I’m currently running into issues with AoT compilation using TVM. My goal is to be able to compile AI models directly into single executables for distribution onto Unix nodes with various architectures. I found that TVM, Glow, and other compilers have the concept of “bundling” although I see that it is most often used for IoT / low-power devices. If I’m using the wrong thing for my use case, please let me know! It’s a bit off to be using MicroTVM and “bundling” for my use case.
I tried following the instructions for compiling TVM for bundling by pulling down TVM as follows:
git clone --recursive https://github.com/apache/tvm tvm
Afterwards, I changed tvm/cmake/config.cmake to enable set(USE_LLVM ON), set(USE_MICRO ON), set(USE_MICRO_STANDALONE_RUNTIME ON). The following is the Dockerfile that I then try to compile TVM with:
# Install latest Ubuntu with Python 3.8.
# Python 3.7.x+ and 3.8.x+ must be used for compatibility with TVM bundling.
FROM ubuntu:22.04
ENV http_proxy $HTTPS_PROXY
ENV https_proxy $HTTPS_PROXY
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update
RUN apt-get install -y software-properties-common && add-apt-repository ppa:deadsnakes/ppa && apt-get update && apt-get install -y python3.8 python3-pip && rm -rf /var/lib/apt/lists/*
# Copy the current directory contents into the container at /app.
WORKDIR /app
COPY . /app
# Install dependencies for TVM.
# Instructions are here for all OS versions: https://tvm.apache.org/docs/install/from_source.html
# Install CMake and enable dev packaging from third-party source: https://apt.kitware.com/
RUN apt-get update
RUN apt-get -y install gpg wget ca-certificates
RUN wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | gpg --dearmor - | tee /usr/share/keyrings/kitware-archive-keyring.gpg >/dev/null
RUN echo 'deb [signed-by=/usr/share/keyrings/kitware-archive-keyring.gpg] https://apt.kitware.com/ubuntu/ jammy main' | tee /etc/apt/sources.list.d/kitware.list >/dev/null
RUN apt-get update
RUN test -f /usr/share/doc/kitware-archive-keyring/copyright || rm /usr/share/keyrings/kitware-archive-keyring.gpg
RUN apt-get install kitware-archive-keyring
RUN apt-get -y install cmake
# Get other TVM dependencies.
RUN apt-get install -y python3 python3-dev python3-setuptools gcc libtinfo-dev zlib1g-dev build-essential libedit-dev libxml2-dev llvm
# Actually build TVM.
WORKDIR /app/tvm
RUN cmake -DCMAKE_BUILD_TYPE=Release -S . -B build
RUN make -j4 -C build
After running the resulting image, I am unable to run make within apps/bundle_deploy and instead get the following error:
ls: cannot access '../../build/standalone_crt': No such file or directory
Makefile:24: *** "CRT not found. Ensure you have built the standalone_crt target and try again". Stop.
I am at a complete loss. Could anyone take a look and see what I’m missing here / if they are able to replicate my issues?
Thank you so much, Oleg