[Tensorize] how to use tensorize for composition op(eg. conv+relu)

weireweire · April 23, 2019, 3:51am

Hi all,
If my hardware can only do the composition op like conv+relu, I can only write this in two compute for tvm.sum must be the top level of a compute.
But tensorize can not deal with composition op yet.
So what can we do in this situation.

zfhn · April 23, 2019, 12:40pm

Is your question similar to the following question？
In tensorize tutorials, if the Hardware support gemv+relu，how to use the feature of hardware instead of doing gemv and relu separately ?

weireweire · April 24, 2019, 1:36am

Yes, and the tutorial only show the case that tensorize a non-composition op.

maplegu · January 15, 2020, 3:31am

I am also looking at a similar use case. Is there any solution or update?

aca88 · February 14, 2020, 9:06am

I am also interested in an update.

@tqchen @yzhliu @zhiics

github.com

apache/incubator-tvm/blob/master/python/tvm/tensor_intrin.py#L123


    raise TypeError("expect Operation")
inputs = op.input_tensors
binds = binds if binds else {}
tensors = list(inputs)
for i in range(op.num_outputs):
    tensors.append(op.output(i))


binds_list = []
for t in inputs:
    if not isinstance(t.op, _tensor.PlaceholderOp):
        raise ValueError("Do not yet support composition op")


cfg = current_build_config()
for t in tensors:
    buf = (binds[t] if t in binds else
           _api.decl_buffer(t.shape, t.dtype, t.op.name,
                            data_alignment=cfg.data_alignment,
                            offset_factor=cfg.offset_factor))
    binds_list.append(buf)


if scalar_params:

adb · February 14, 2020, 9:11pm

+1 for my team. Current work around is using pragmas for hints about fusion at codegen stage

xwrock · March 24, 2020, 9:33am

Hi, can you describe your schemes a bit more thoroughly? I encounted this problem as well.

adb · March 24, 2020, 5:29pm

@xwrock For now I think the proper way to work around this is to go the BYOC route until this part of TVM becomes more flexible for custom accelerators.

aca88 · April 6, 2020, 8:33am

@xwrock another way is to bypass tensorize like the ppl of VTA did. It requires some manipulation of the low level AST, but I guess the end effect are the same.

JosseVanDelm · March 29, 2021, 1:36pm

@aca88 Looks like i’m running into the same issue Can you point to the code of VTA where they do this tensorization bypass?

Thanks!