Thanks for the discussion! It’s indeed quite insightful and I personally learned a lot 
Just wanted to share my 2 cents:
Library reloading is clean. I completely agree, and if there is no much overhead, on-demand library loading could be even better if TVM is integrated with a frontend framework (PyTorch) which ships with its own LLVM.
Compilation speed matters. In our auto-tuning process, compilation is usually invoked more than 20k times, which is a major bottleneck. Therefore, if library reloading impacts performance, it would be less efficient for AutoTVM/AutoScheduler/MetaSchedule.
LLVM’s ongoing effort. As @kparzysz said, LLVM seems to be trying to fix this in 15.0.0. Well, it’s a bit slow, but given LLVM is moving towards this direction, I believe some day in the distant future this feature could be turned on without having to resort to library reloading.
My proposal. Given TQ’s approach seems to suffice already, I would say we could move forward with his proposal for now. If in the end we still need library reloading, we might make it an optional environment variable which could be turned on or off given the need for fast compilation and static linking.