Compositionality is a fundamental philosophy so please allow me to elaborate a bit more here. One great example to show its importance is the design of deep learning frameworks.
In a deep learning framework. The concept of layers is composable. I can take a residual layer, compose it with softmax loss function and optimizer loss. These layers are abstracted under a common interface nn.module. Each Module transforms an object of interest – Tensor.
Tensor itself can be viewed as containing certain C1-type constraint information. Such as the shape, the data content, the device it resides in. Some of the layers (e.g. CuDNNLayer) may only work under the constraint of a gpu device.
Importantly, there are also C0-type configurations. For example, the number of hidden neurons or stages of residual connections. These information are not part of Tensor, because it is sufficient to apply those transformations and Tensor itself contains minimum but sufficient information for followup layers to apply further transformations. Applying more information to the Tensor could create more constraints, and possibly confusion about how to handle those attributes.
Deep learning frameworks are maximally composable; we can compose a residual block with a classification loss(softmax) or detection loss to form different models. These layers can be developed by different developers.
In summary, composability is obtained by decoupling information and clarifying the minimum but necessary information in the key data structure of interest.
Come back to the case of TVM. IRModule is a bit like Tensor, and we are talking about putting all the configurations of the layers, as well as the device information into a centralized place. If we are only looking at one official model(say resnet-256), this of course can help clarify all the options available, but would restrict the evolution to a single pipeline(and forcing all the options into that one place). A deep learning framework approach would be to only keep minimum information(C1-type) in the key data structure, allowing C0-type separately. For specific applications, there might be a centralized configuration(e.g. argparse) which informs C0 type config, but that centralized config is not part of the Tensor.
In summary, putting all the configurations(C0 and C1 kinds) into a single place will certainly improve clarity if we only have a single pipeline in mind. The C0-type configuration brings un-necessary and sometimes confusing information. Remember that pass writers generally need to take the constraints in IRModule seriously, having C0-type information in IRModule would make developers wonder whether or not they should be considered (which is an open set as we grow the set of passes).
In the spirit of minimum but sufficient principle, we want to limit the information attached to IRModule to C1-type information. Note that this does not preclude that a high-level single pipeline builds a centralized config which then propagates to the lower-level mechanism. I believe that was the original intention, and the main reason I brought up compositionality is that at the level of IRModule and pass we will need to consider such separately carefully.
Since the topic of compositionality is quite important. Let us also study a few more examples:
Example 0: Imagine that we want to try out the following thing. stage0: different kinds of unroll factors or vectorization factors, benchmark, do them in a loop then compare the result; stage1: send to the followup lowering optimizations with another set of configs. In this case C0-type config(e.g. unrolling-factor) in stage0 is not relevant to stage1. Given the collection of different choices of stage0 in a loop, it is also not entirely desirable or possible to centralize the configs into a single data structure for this case.
Example 1: Imagine that people want to build alternative compilation pipelines with a different set of configs(e.g. Running through quantization then building). In this case it may not be desirable to couple the two types of config together, since each pipeline may only care about one set.
We can find that most of the examples come from alternative optimization pipelines and choices that may be different from the current build pipeline. These are however, important cases to support so they either can be incorporated into future pipelines, or simply enable more preprocessing choices that composes with the build pipeline.
Compositionality does not imply that we need to ask everyone to use the same config for all possible pipelines . Instead, the main implication is to clarify what is minimum but necessary (and needs to be considered by all passes/pipelines), while leaving out other parts, so it leaves space of flexibility to others.
Coming back to the particular topic of this RFC. I think we acknowledge that it could be useful to have a centralized config for a single tvmc pipeline which can help to bring the clarity. We also agree that the discussion is not about changing the set of information, but mainly about how we organize the infomation. The main point of compositionality is to carefully dissect the two kinds of configurations when it comes to putting the information in IRModule and how do the two kinds of configurations interact with passes.
