Downgrade TVM Runtime to C++11?

Development
#1

Recently, we updated the TVM dependencies to require C++17 TVM is now using C++17

We ran into a situation where we want to deploy libtvm_runtime to a system with an older glibc (Suse and Redhat still ship really old versions of gcc).

I love the use of C++17 in the compiler itself, but how to people feel about maintaining the older standards for the runtime to ease deployment? If we could require that the runtime build with C++11 and allow the main compiler to build with C++17, I think that hit a sweet spot in development and deployability.

cc @tqchen @rkimball @kparzysz @leandron @areusch

1 Like
[TVM BYOC] compilation error while compiling CUSTOM codegen generated C file
#2

The TVM runtime hasn’t been compatible with C++11 for a while (long before the switch to C++17).

At this point I’m opposed to such downgrades. The users of older platforms can always download and install newer versions of C++ compilers, either gcc or clang.

#3

I’m sort of torn. On the one hand, I understand that upgrading a compiler shouldn’t be a big burden. On the other hand, the person facing the need to upgrade may not be immediately in a good position to compile/build a whole compilation toolchain for a particular platform.

I know we’ve been using the c++14 compiler flag for a while, but was libtvm_runtime.so in fact actually c++11-compatible til recently? That might be helpful context to add here.

#4

I know things can get hairy with ABI backward compatibility but does BUILD_STATIC_RUNTIME help here at all?

Having a single compiler standard across the whole project definitely eases things from a development perspective so we should keep it if we can. The docs (which have been recently updated) specify GCC 7.1+ is required, has much of the runtime been updated to use any C++17/14 features?

#5

A workaround to this problem to add this line add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0) to tvm’s CMakeList.txt and make sure your host app also compiled with this option, to force tvm and host app use same non-C11 ABI. It works well in our environment.

#6

If there are no many changes requiring C++14/17 and we can clean them out from the code, it might have sense to do this because runtime is assumed very lightweight which can be deployed to everywhere. In opposite to the tvm compiler itself. The question is - how many changes will require. + need to add at least build in CI with such limitations.

#7

I still think C++11/14 for runtime is a safer choice as I posted on Aug LLVM is moving to C++17 - #17 by FrozenGene. For deployment environment, developers can not decide to upgrade compilers or change the deployment environment most of time. Compiler / Runtime could be different C++ language std from my view. Industry deployment environment is complex and somehow a little behind the latest trend.

#8

@mbrookhart : Could you say a little more about which versions of RHEL / Suse / GCC people want to use?

I imagine our options depend a bit on just how old we’re talking about.

#9

Seconding @cconvey’s question on the specific version of RHEL to be supported. In addition, is the goal to deploy onto a specific RHEL version, or to deploy onto a specific RHEL version without access to root? For the former, after playing around a bit in a CentOS docker image, installing devtoolset-* makes it straightforward to use newer versions of g++.

[root@f2059d87d7b8 /]# scl enable devtoolset-11 -- g++ --version
g++ (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[root@f2059d87d7b8 /]#

If deploying to locations without package installation privs, it looks like other packages require a newer version of gcc (e.g. a 2020 thread about pytorch not supporting gcc 4.8, which is RHEL 7’s default), but may package a compatible libstdc++.so for these platforms. Since RHEL 7 is still considered production support, but is limited to gcc 4.8, I’d lean that route rather than maintaining support for a compiler from 2013.

Unfortunately, from the page on gcc’s ABI policy and this stackoverflow post, while glibc is forward compatible with newer gcc versions, newer gcc versions cannot target an older glibc version. I did some tests looking at the specific symbols that are used (command below if anybody wants to run a more thorough search), crossing my fingers that we’d get lucky and just happen to use only symbols available in older gcc versions. But, that isn’t the case for either g++ 10.3 (my local machine, produces libtvm_runtime.so that requires GLIBCXX 3.4.26) or g++ 7.5.0 (current CI version, produces libtvm_runtime.so that requires GLIBCXX version 3.2.22).

nm --demangle --extern-only --dynamic --undefined-only build/libtvm_runtime.so \
    | grep GLIBCXX \
    | sort --version-sort --field-separator=@ --key=2

Based on RHEL version lifetime, the oldest production-supported RHEL version is RHEL 7, which provides gcc 4.8, which provides GLIBCXX version 3.4.18.

(Apologies for the link-spam in this post, but figuring out exact versions required enough tracking down that I wanted to record them somewhere.)

#10

Take a look at this Red Hat Developer Toolset Overview | Red Hat Developer . IIRC this is newer gcc’s built against older glibc but installable on RHEL.

#11

My apologies for the delay in response. GCC 4.8 is indeed the oldest default I’m aware of in still-supported distributions. gcc 6.3 and 7.5 are also common. GCC 6.3 can support C++14, but not 17.

I know redhat supports newer compilers with older glibc, my main concern is around cross-compilation.

#12

It’s not turning out to be a hard requirement, we’re finding ways to work around it, but I don’t think we’re blocked.

#13

So far it seems like while there is some concern about getting into a situation where an older OS doesn’t have a C++17-compatible toolchain, we’ve mainly been able to work around it. We do have the C runtime as an alternative, and have documented that through bundle_deploy iirc. You wouldn’t be able to do heterogenous execution with the C runtime, but I wonder if it’s likely someone would be using such an old OS with e.g. a GPU in the picture.

#14

I think hearing no protests here, the conclusion of the thread is that downgrading is not required. I will proceed to merge https://github.com/apache/tvm/pull/12691 which will further push us down the path of requiring C++17 in runtime.

#15

Sounds good. Thanks, everyone!