[AutoTVM] Defining a search space

Questions
1

In the Tune Conv2d we define a search space using a config, where all the tuneable parts are defined.

Then in the Auto-tuning a convolutional network tutorial we get a test network from nnvm and tune that. So, is this network already defined with a config and template functions?
If so, are these tuning tasks then extracted from the graph when we call autotvm.task.extract_from_graph() on the graph?

I noticed the custom network in the second tutorial is built using NNVM functions. Does this mean that if I import a model graph from keras, I could still tune this graph using a similar method?

Does this currently only apply to Convolution and Depthwise Convolution operators? Is this because these are the only operators currently implemented in a way to support this?

Does the tuning take into account any potential tuning dependencies between operators, or is each treated as an independent instance?

1 Like
2

Then in the Auto-tuning a convolutional network tutorial we get a test network from nnvm and tune that. So, is this network already defined with a config and template functions?

Sort of, the network is already defined in nnvm, and the operators for each layer already have templates. Workload extraction matches the nnvm layers with tvm templates and gives them concrete shapes.

I noticed the custom network in the second tutorial is built using NNVM functions. Does this mean that if I import a model graph from keras, I could still tune this graph using a similar method?

Yes, if the operators and hardware combination you want to tune for already have templates. Currently the most well supported operators in terms of templates are conv2d and depthwise_conv2d.

Does the tuning take into account any potential tuning dependencies between operators, or is each treated as an independent instance?

Currently tuning is independent for each operator and shape, although we are currently working on graph level tuning (e.g., for data layout dependencies) see #1585.

3

For conv2d, its space is defined in https://github.com/dmlc/tvm/blob/master/topi/python/topi/arm_cpu/conv2d.py

4

Thanks for the replies!

Why do we need separate templates for the same operator?

5

In some cases, the same operator may be implemented with a different algorithm, such as im2col vs. direct convolution vs. winograd. Even with Halide’s separation of schedules from declarations, a different algorithm currently requires a different declaration, which defines a different search space (and template).