in implementing the draft PR and collecting the various package constraints from the codebase, i’ve now come across several different cases where we may want to add version constraints that aren’t strictly necessary for functional reasons. It’s not clear that these constraints should live in the gen_requirements.py tool from the PR. in scattering various pip3 install commands around the codebase, we’ve implicitly added this context, but we haven’t always documented why we’re restricting the version or the context for the restriction.
here are some contexts:
-
Functional Constraints – in some cases e.g.
synr,xgboost, we must constrain TVM to a particular min version to avoid bugs. as we check these in tomain, these will always be min version constraints and never max version constraints (an example of a max version constraint would be if we were making a release and discovered e.g. a just-released pytorch is incompatible and we have no time to patch–in this case, we want a release constraint, see below). -
Dev Constraints – some packages e.g.
pylint,cpplint,blackare purely used as developer tools. for best developer experience, their output should be the same across all installations. therefore, those packages and their dependencies should be pinned exactly to whatever version is used inci-lint. however, it should be possible to easily rebuildci-lintto use the latest version. -
Tested Constraints – people using TVM from the source tree should be able to install a set of dependencies that are the same version as is tested in the CI. However, these aren’t
ci-constraints.txtas mentioned in this proposal–we don’t want to constrain future CI container builds, we just want to document and use whatever was selected at the previous CI container build. -
Release Constraints – when building a tlcpack pip package, we may want to decide which of these constraints to impose on the generated wheels. for example, we may want to produce a pip package with no version constraints and one with CI-tested versions.
i’d propose to move forward with the current PR adding the Functional Constraints only to gen_requirements.py. However, the current PR doesn’t accommodate the remaining use scenarios and I don’t think we’ve fully addressed them here. Before we can address the remaining contexts, we’ll need to consider how all of this work is reflected in the CI–so, I’d like to merge this PR as an improvement to setup.py, and then work towards a follow-on RFC/PR which proposes changes to the process by which we build and update CI containers.