Benefits of cythonization in tvm?

LeslieGerman · July 3, 2020, 3:51pm

Hi all,

TVM python code can be compiled with cython on Linux:

apache/incubator-tvm/blob/575a3835315a533a19e871ee913f01142befebab/python/setup.py#L64


                    libs.append(name)
                    break
    else:
        libs = None
    return libs, version


LIB_LIST, __version__ = get_lib_path()


def config_cython():
    """Try to configure cython and return cython configuration"""
    if os.name == 'nt':
        print("WARNING: Cython is not supported on Windows, will compile without cython module")
        return []
    sys_cflags = sysconfig.get_config_var("CFLAGS")

    if "i386" in sys_cflags and "x86_64" in sys_cflags:
        print("WARNING: Cython library may not be compiled correctly with both i386 and x64")
        return []
    try:

In fact it is automatically activated when cython is installed.

Can anyone please tell me what advantages this brings for tvm?

The generic assumption is that cythonized code supposed to be faster, but sometimes the gain is not significant, or even the code can be slower.
Does anyone have any benchmarks, or did any measurements which show if cythonized TVM code is faster, how much faster, and in which cases?

Thanks!

tqchen · July 3, 2020, 4:42pm

We are not cythonizing all the python code.

TVM is primarily written in C++ and we interact with Libtvm via a PackedFunc FFI defined by a stable C API. We provide two ways to interact with the C API: ctypes and cython.

You can view the cython part as a way to interact with the python C API. Most of the python APIs are then build on top of the FFI and are written in python.

Cythonized ffi provides lower ffi overhead than the ctypes variant and can get close to the overhead of numpy api calls

LeslieGerman · July 6, 2020, 11:32am

Thanks for detailed explanation!