Meta, hell-bent on catching as much as rivals within the generative AI space, is spending billions by itself AI efforts. A portion of those billions goes toward recruiting AI researchers. But an excellent larger chunk is being spent developing hardware, specifically chips to run and train Meta’s AI models.
Meta unveiled the latest fruit of its chip dev efforts today, conspicuously a day after Intel announced its latest AI accelerator hardware. Called the “next-gen” Meta Training and Inference Accelerator (MTIA), the successor to last yr’s MTIA v1, the chip runs models including for rating and recommending display ads on Meta’s properties (e.g. Facebook).
In comparison with MTIA v1, which was built on a 7nm process, the next-gen MTIA is 5nm. (In chip manufacturing, “process” refers to the scale of the smallest component that may be built on the chip.) The subsequent-gen MTIA is a physically larger design, full of more processing cores than its predecessor. And while it consumes more power — 90W versus 25W — it also boasts more internal memory (128MB versus 64MB) and runs at a better average clock speed (1.35GHz up from 800MHz).
Meta says the next-gen MTIA is currently live in 16 of its data center regions and delivering as much as 3x overall higher performance in comparison with MTIA v1. If that “3x” claim sounds a bit vague, you’re not unsuitable — we thought so too. but Meta would only volunteer that the figure got here from testing the performance of “4 key models” across each chips.
“Because we control the entire stack, we will achieve greater efficiency in comparison with commercially available GPUs,” Meta writes in a blog post shared with TechCrunch.
Meta’s hardware showcase — which comes a mere 24 hours after a press briefing on the corporate’s various ongoing generative AI initiatives — is unusual for several reasons.
One, Meta reveals within the blog post that it’s not using the next-gen MTIA for generative AI training workloads for the time being, although the corporate claims it has “several programs underway” exploring this. Two, Meta admits that the next-gen MTIA won’t replace GPUs for running or training models — but as a substitute complement them.
Reading between the lines, Meta is moving slowly — perhaps more slowly than it’d like.
Meta’s AI teams are almost definitely under pressure to chop costs. The corporate’s set to spend an estimated $18 billion by the tip of 2024 on GPUs for training and running generative AI models, and — with training costs for cutting-edge generative models ranging within the tens of tens of millions of dollars — in-house hardware presents a lovely alternative.
And while Meta’s hardware drags, rivals are pulling ahead, much to the consternation of Meta’s leadership, I’d suspect.
Google this week made its fifth-generation custom chip for training AI models, TPU v5p, generally available to Google Cloud customers, and revealed its first dedicated chip for running models, Axion. Amazon has several custom AI chip families under its belt. And Microsoft last yr jumped into the fray with the Azure Maia AI Accelerator and the Azure Cobalt 100 CPU.
Within the blog post, Meta says it took fewer than nine months to “go from first silicon to production models” of the next-gen MTIA, which to be fair is shorter than the standard window between Google TPUs. But Meta has numerous catching as much as do if it hopes to realize a measure of independence from third-party GPUs — and match its stiff competition.
