Why Relay.vm.compile takes hours?

Questions
1

Hi

I am trying to compile a model with llvm target, and it takes several hours for the llvm codegen part. Is there any way to accelerate it?

2

I guess some passes have problem. How about:

        with relay.build_config(opt_level=0):
            graph, lib, _params = relay.build(...)

Opt level 0 almost disable all passes.

3

Hi:

Thanks a lot, but changing opt_level does not fix this problem. When opt_level <= 1, no cached functions will be generated int vm.compiler.Lower. And when ‘opt_level >= 2’, the TVM will still stuck in the codegen::Build part.

Here is some logs looks odd

[01:22:05] /root/Codes/tvm_in_mac/src/te/schedule/bound.cc:119: not in feed graph consumer = hybrid(_broadcast_shape_func, 0x55efa43d1250)
[01:22:05] /root/Codes/tvm_in_mac/src/te/schedule/bound.cc:119: not in feed graph consumer = hybrid(_broadcast_shape_func, 0x55efa61a4190)
[01:22:05] /root/Codes/tvm_in_mac/src/te/schedule/bound.cc:119: not in feed graph consumer = hybrid(_squeeze_shape_func, 0x55efa619a210)
[01:22:05] /root/Codes/tvm_in_mac/src/te/schedule/bound.cc:119: not in feed graph consumer = hybrid(_broadcast_shape_func, 0x55efa43d5390)
[01:22:05] /root/Codes/tvm_in_mac/src/te/schedule/bound.cc:119: not in feed graph consumer = hybrid(_broadcast_shape_func, 0x55efa61acda0)
[01:22:05] /root/Codes/tvm_in_mac/src/te/schedule/bound.cc:119: not in feed graph consumer = hybrid(_split_shape_func, 0x55efa4ea7850)
[01:22:05] /root/Codes/tvm_in_mac/src/te/schedule/bound.cc:119: not in feed graph consumer = hybrid(_split_shape_func, 0x55efa61b10f0)
[01:22:05] /root/Codes/tvm_in_mac/src/te/schedule/bound.cc:119: not in feed graph consumer = hybrid(_broadcast_shape_func, 0x55efa6159a50)
[01:22:05] /root/Codes/tvm_in_mac/src/te/schedule/bound.cc:119: not in feed graph consumer = hybrid(_split_shape_func, 0x55efa5031ec0)
[01:22:05] /root/Codes/tvm_in_mac/src/te/schedule/bound.cc:119: not in feed graph consumer = hybrid(_split_shape_func, 0x55efa4ea6400)
4

Hi:

It seems that a MatMul operation with strange input shapes cause this problem, and I think the schedule function of dense can not handle this situation. Besides, cublas works fine to deal with such situation.

The graph_def can reproduce this problem

debug_graph = tf.Graph()
with debug_graph.as_default():
    input_1 = tf.placeholder(dtype=tf.float32, shape=[215296, 1], name='input_1')
    input_2 = tf.placeholder(dtype=tf.float32, shape=[1, 4], name='input_2')
    result = tf.matmul(input_1, input_2, name='result')
5

Do you mind providing one .pb file or relay api to reproduce? Your code snippet can not be used directly. I would like to spend some time investigating it.

6

Sure, you can just copy these codes into .py file to reproduce the problem.

import tvm
import tensorflow as tf
from tvm import relay

def main():
    # create graph
    debug_graph = tf.Graph()
    with debug_graph.as_default():
        input_1 = tf.placeholder(dtype=tf.float32, shape=[215296, 1], name='input_1')
        input_2 = tf.placeholder(dtype=tf.float32, shape=[1, 4], name='input_2')
        result = tf.matmul(input_1, input_2, name='result')

    # create ir module
    layout = "NHWC"
    mod, params = relay.frontend.from_tensorflow(
        debug_graph.as_graph_def(),
        layout=layout,
        outputs=['result']
    )

    # 
    target = "llvm"
    context = tvm.cpu()
    exe = relay.vm.compile(mod, target=target, params=params)


if __name__ == '__main__':
    main()