Prior to being acquired by Broadcom, VMware had been a long-time academic sponsor of UC Berkeley’s Sky Computing Lab. Our collaboration came naturally because we were both passionate about tech interoperability and innovating with pragmatism to solve pressing industry challenges. Of course, Broadcom’s history is also centered on driving innovation and interoperability through technologies such as Ethernet. That’s why we are excited to share that we are expanding our research sponsorship of UC Berkeley’s Sky Computing Lab, starting with a $4 million dollar donation to further accelerate the lab’s mission.
To appreciate our excitement about the research Broadcom is sponsoring, it helps to consider UC Berkeley’s history of invention and innovation excellence. Their team that built Apache Spark went on to found Databricks. More recently, projects such as Ray, vLLM, and Chatbot Arena have become central to the generative AI ecosystem.
- Ray is an open-source distributed computing framework that allows developers to distribute workloads across multiple CPUs, GPUs, or even clusters of machines with minimal code changes. Ray has been used to drive massive scale AI environments, including those at OpenAI.
- vLLM is an open-source model serving system designed to optimize the inference performance of large language models (LLMs). vLLM has emerged as the most popular open source AI inference engine today.
- Chatbot Arena is the most popular service for comparing and ranking chatbots and LLMs, with its LLM Leaderboard serving as the de facto place to compare popular LLMs.
We are equally excited at the promise of several of Sky Computing Lab’s emergent projects with the intent to comprise a complete open-source interoperability software stack called Sky AI Software Infrastructure (SAISI). Think of SAISI as bringing projects together that allow your models and applications to consistently operate across a variety of accelerators. The SAISI projects that provide the required interoperability interconnects are highlighted in blue below.
When one considers open-source AI interoperability today, there are many projects to choose from, but what is required for us to collectively move forward is consensus around a unified interoperability stack, and that’s what makes SAISI so compelling. Beyond the projects previously mentioned, here’s why the other SAISI projects matter:
- NovaSky is a relatively new project with the goal of building a highly flexible and efficient framework for post-training workloads. We are continuing to see growing traction for domain-specific expert models, and this will make model post-training a key part of the AI strategy in all organizations across all industries.
- UCCL aims to build an efficient collective communication library for hardware accelerators, offering (1) faster collectives through multi-path communication; (2) broad cloud availability by supporting legacy NICs and Ethernet fabrics; (3) evolvable transport design with support for multi-path load balancing and congestion control; (4) an open-source platform for research on ML collectives.
- SkyPilot aims to abstract away the complexity and differences across clouds in managing instances and containers. This is critically important as hybrid AI use cases are beginning to take hold, such as when organizations desire to train or fine tune a model in a public cloud and deploy the model for inference at an enterprise data center or edge site.
Ion Stoica, director of the Sky Computing Lab, shared his enthusiasm about the collaboration: “VMware has been one of our strongest and most reliable collaborators over the past decade, and we’re thrilled to see this relationship continue and grow following its acquisition by Broadcom. We look forward to collaborating with Broadcom engineers to build an open-source AI stack that powers the next generation of applications and services by efficiently leveraging the growing diversity of hardware accelerators and network technologies.”
Beyond our financial donation, Broadcom’s very best engineers will expand our collaboration with the Sky Computing Lab by volunteering their time to advise the Lab’s researchers based on our experience building and scaling large open ecosystems. The pace of AI innovation is continuing to accelerate, making meaningful partnerships with shared goals even more important. I’m excited to see how UC Berkeley’s talented faculty and researchers will continue to make high-impact technology contributions to the AI ecosystem, and I encourage Broadcom’s industry partners to also donate their money and time to the Sky Computing Lab to further shape the open-source technology that is essential to the growth of our industry.