A yr ago, Databricks acquired MosaicML for $1.3 billion. Now rebranded as Mosaic AI, the platform has change into integral to Databricks’ AI solutions. Today, at the corporate’s Data + AI Summit, it’s launching a variety of recent features for the service. Ahead of the announcements, I spoke to Databricks co-founders CEO Ali Ghodsi and CTO Matei Zaharia.
Databricks is launching five recent Mosaic AI tools at its conference: Mosaic AI Agent Framework, Mosaic AI Agent Evaluation, Mosaic AI Tools Catalog, Mosaic AI Model Training, and Mosaic AI Gateway.
“It’s been an awesome yr — huge developments in Gen AI. Everybody’s enthusiastic about it,” Ghodsi told me. “However the things everybody cares about are still the identical three things: how will we make the standard or reliability of those models go up? Number two, how will we make sure that that it’s cost-efficient? And there’s an enormous variance in cost between models here — a huge, orders-of-magnitude difference in price. And third, how will we do this in a way that we keep the privacy of our data?”
Today’s launches aim to cover nearly all of these concerns for Databricks’ customers.
Zaharia also noted that the enterprises which can be now deploying large language models (LLMs) into production are using systems which have multiple components. That usually means they make multiple calls to a model (or perhaps multiple models, too), and use quite a lot of external tools for accessing databases or doing retrieval augmented generation (RAG). These compound systems speed up LLM-based applications, lower your expenses through the use of cheaper models for specific queries or caching results and, perhaps most significantly, make the outcomes more trustworthy and relevant by augmenting the inspiration models with proprietary data.
“We predict that’s the long run of really high-impact, mission-critical AI applications,” he explained. “Because in case you give it some thought, in case you’re doing something really mission critical, you’ll want engineers to have the option to regulate all features of it — and also you do this with a modular system. So we’re developing a number of basic research on what’s the perfect strategy to create these [systems] for a particular task so developers can easily work with them and hook up all of the bits, trace the whole lot through, and see what’s happening.”
As for actually constructing these systems, Databricks is launching two services this week: the Mosaic AI Agent Framework and the Mosaic AI Tools Catalog. The AI Agent Framework takes the corporate’s serverless vector search functionality, which became generally available last month and provides developers with the tools to construct their very own RAG-based applications on top of that.
Ghodsi and Zaharia emphasized that the Databricks vector search system uses a hybrid approach, combining classic keyword-based search with embedding search. All of that is integrated deeply with the Databricks data lake and the information on each platforms is at all times routinely kept in sync. This includes the governance features of the general Databricks platform — and specifically the Databricks Unity Catalog governance layer — to make sure, for instance, that non-public information doesn’t leak into the vector search service.
Talking in regards to the Unity Catalog (which the corporate is now also slowly open sourcing), it’s price noting that Databricks is now extending this method to let enterprises govern which AI tools and functions these LLMs can call upon when generating answers. This catalog, Databricks says, can even make these services more discoverable across an organization.
Ghodsi also highlighted that developers can now take all of those tools to construct their very own agents by chaining together models and functions using Langchain or LlamaIndex, for instance. And indeed, Zaharia tells me that a number of Databricks customers are already using these tools today.
“There are a number of firms using this stuff, even the agent-like workflows. I feel individuals are often surprised by what number of there are, nevertheless it appears to be the direction things are going. And we’ve also present in our internal AI applications, just like the assistant applications for our platform, that that is the strategy to construct them,” he said.
To judge these recent applications Databricks can also be launching the Mosaic AI Agent Evaluation, an AI-assisted evaluation tool that mixes LLM-based judges to check how well the AI does in production, but additionally allows enterprises to quickly get feedback from users (and allow them to label some initial data sets, too). The Quality Lab features a UI component based on Databricks’ acquisition of Lilac earlier this yr, which lets users visualize and search massive text data sets.
“Every customer we’ve is saying: I do have to do some labeling internally, I’m going to have some employees do it. I just need perhaps 100 answers, or perhaps 500 answers — after which we are able to feed that into the LLM judges,” Ghodsi explained.
One other strategy to improve results is through the use of fine-tuned models. For this, Databricks now offers the Mosaic AI Model Training service, which — you guessed it — allows its users to fine-tune models with their organization’s private data to assist them perform higher on specific tasks.
The last recent tool is the Mosaic AI Gateway, which the corporate describes as a “unified interface to question, manage, and deploy any open source or proprietary model.” The concept here is to permit users to question any LLM in a governed way, using a centralized credentials store. No enterprise, in any case, wants its engineers to send random data to third-party services.
In times of shrinking budgets, the AI Gateway also allows IT to set rate limits for various vendors to maintain costs manageable. Moreover, these enterprises then also get usage tracking and tracing for debugging these systems.
As Ghodsi told me, all of those recent features are a response to how Databricks’ users are actually working with LLMs. “We saw an enormous shift occur available in the market within the last quarter and a half. Starting of last yr, anyone you talk over with, they’d say: we’re pro open source, open source is awesome. But once you really pushed people, they were using Open AI. Everybody, irrespective of what they said, irrespective of how much they were touting how open source is awesome, behind the scenes, they were using Open AI.” Now, these customers have change into way more sophisticated and are using open models (only a few are really open source, after all), which in turn requires them to adopt a wholly recent set of tools to tackle the issues — and opportunities — that include that.
