I think the functionality all makes sense to me. I have some questions on the implementation:
- Why do you assign global variables when declaring a param, then shadow them in those functions that use them? This seems like a bad design because unparameterized functions don’t get an unbound variable error if they try to use a param.
size = tvm.testing.parameter(1, 2, 3) - It would be great to qualify
cache=True, for examplecache_return_value=Trueor something. - What’s the procedure for reproducing an error in a test case with several parameter axes which does not occur when
TVM_TEST_DISABLE_CACHEis provided and the test cases are filtered such that only 1 parameterization runs? - Does it make sense to impose any requirements on the cached values e.g. cached fixtures must return serializable values, such that reproducing the above scenario is likelier to be easier?
- Can you give some examples from existing tests of the kinds of things you expect to be cached? I know we cache some testdata, which is clearly serializable. Are there other examples which could help inform a decision about 4? I think either way, it would be great to motivate the type of cache with some examples of things that we expect to cache.