Hi, I am working on heterogeneous execution, but I don’t quite understand why we need this “with target”. For a single target, we can do this, but it is not working for multiple ones. I think we might use it to pass the target somewhere in python for lowering or finding schedules. But shouldn’t we pass the target from GraphCompile since we’ve also attached it there?
Best,
Zhi
Graph compile calls into topi functions, and most topi function does not accept target as an explicit argument. So we need to do with target scope at some point before calling into these functions
@tqchen I see. Thanks. Do you have any suggestion on heterogeneous execution? The problem is that we sometimes cannot find the correct schedule because we always only specify one target here. Do we need to refactor the topi functions and pass the target to them if there don’t have?
We don’t have to refactor topi, just have to call with target at a later time, maybe when we invoke the compute and schedule per operatorb
Thanks. That’s something I was trying. But with @yzhliu 's help, we realized that we have with.target in schedule (e.g. https://github.com/dmlc/tvm/blob/master/nnvm/python/nnvm/top/nn.py#L136), but we don’t have that in compute, because we didn’t pass it from fcompute in compile_engine.cc (https://github.com/dmlc/tvm/blob/master/nnvm/src/compiler/compile_engine.cc#L202).
Shouldn’t it make more sense to pass from there and do “with.target” the same as schedule in a later stage when we need it?
Thanks,
Zhi
To make it more clear, I think it would be necessary to keep interfaces of compute and schedule consistent. But that may need all registration of FTVMCompute to take the “target” field.
Another way in my mind is to attach the device name in the nodeattr. We can get the target information based on the device in compute. But we probably need to keep the “with target” in the build_module.py because the device information is unavailable if the annotation pass is not applied.
@Laurawly, @yzhliu Do you have any comments or concerns? Thanks.
Can we always have the annotation pass, but for Homogeneous case, simply tag with the same target passed in. I feel uncomfortable about the “with target” in build_module.py
With target seems unclear for me as well. I also encountered cases where without using with target in tvm I got wrong compute/schedule.
If later we’re going to add ops which need target info except conv2d currently, we may still encounter the same problem here. So making them consistent sounds reasonable.
@yzhliu My feeling is that we should have the flexibility to choose whether to use the annotation pass or not for the homogeneous case. Otherwise, we can always apply annotation and get target in compute.