As we all have witnessed in the past half year, the world of AI/ML is evolving rapidly, with the arrival of foundational models: stable diffusion for image generation, whisper for voice recognition, GPT, and open LLMs(llama, vicuna, MPT, RedPajama, Falcon).
These pipelines encapsulate unique characteristics in aspects like: memory cost, demand for extreme quantization, and dynamic shape becoming a mandatary. They also involve more complex pipelines. Things are no longer a single tensor in-out situation, and there is a great demand for more customization on how optimization, quantization mechanisms, and deployments are done.
These factors would bring the need for innovations and solutions that solves these problems, sometimes in different ways from the existing workloads we tackled before. Additionally, the rate of innovations in the foundational models is high, bringing the need for rapid evolution of solutions to meet the modeling optimization needs for the modules related to foundational models.
All of these factors bring a need for open-source ML framework communities to think about the strategies. While it is not necessary to re-orient everything that we are doing towards foundational models, certainly, there is a great need to have a strategy to enable and amplify thrusts in the TVM community for foundational models – a lot of that also requires us to bring in innovative solutions to the TVM umbrella and support these emerging needs.
Of course, as an open community, this is also a choice up to the community. I am opening up this thread to see your thoughts and also think it is quite important to our relevance in the ML/AI ecosystem in general.
This is a discussion thread to see community member’s intent. Specifically, would you like the tvm community to support and focus thrusts to innovate foundational models(that of course, is not exclusive to other areas that we already support).
- Yes, love to support community to enable and amplify thrusts for foundational models
- I don’t see a strong need
Please share your thoughts and suggestions in reply I also open a short poll to see our overall interests