Together AI raises $800M to grow its AI-optimized public cloud

Together AI Inc., the operator of a cloud platform optimized to run open-source artificial intelligence models, has raised $800 million from investors.

The startup stated in its funding announcement today that the Series C deal was led by Aramco Ventures. Nvidia Corp., Vista Equity Partners, General Catalyst and a number of other other institutional backers contributed as well. Together AI is now value $8.3 billion.

Together AI’s platform features a serverless inference service that developers can use to run open-source AI models, which removes the necessity to configure graphics cards and network equipment. It claims its serverless environments provide about twice the performance of the fastest alternative.

The corporate also sells three other inference services. Two use dedicated infrastructure that gives more reliability guarantees and customization options than its serverless offering. The third service, Batch Inference, prioritizes cost-efficiency over speed. It provides an as much as 50% price reduction for models that don’t require the flexibility to reply user prompts immediately.

Under the hood, Together AI’s platform is powered by Nvidia chips and a custom software engine called ATLAS. It uses a machine learning technique called speculative decoding to hurry up customer workloads.

Speculative decoding enables engineers to integrate their AI model with a second, lighter neural network. When a user enters a prompt, the lighter algorithm quickly generates a draft response. The principal model then checks it for errors, makes any changes which may be mandatory and delivers the prompt response to the user. That process is considerably faster than having the principal model generate the output by itself.

Normally, the lightweight algorithm that creates draft responses has a hard and fast configuration. Models with a hard and fast configuration often develop into less accurate over time. In response to Together AI, its ATLAS technology addresses the difficulty by routinely adapting the lightweight model to changes in user requirements. The corporate claims its software can speed up some inference workloads by 400%.

Customers can even use Together AI’s platform to fine-tune open-source models. It provides access to training clusters with as much as hundreds of graphics cards. Developers can manage the clusters using Kubernetes, which is comparatively easy to make use of, or a tool called Slurm that gives more customization options.

Considered one of the principal challenges involved in AI training projects is that graphics cards sometimes experience technical issues. In some cases, chip failures can introduce errors into the training workflow. Together AI’s training clusters include software that routinely detects and remediates technical issues.

The corporate disclosed today that its annual bookings topped $1.15 billion within the second quarter. Its platform is utilized by several thousand organizations including LG Inc.’s AI research lab, Cohere Inc. and the Mozilla Foundation. 

Together AI will use its newly raised capital to purchase more infrastructure. It hopes to grow its public cloud’s capability by an element of fifty over the  next five years. As well as, it plans to reinforce its training and inference features.

Photo of Together AI’s founders: Together AI