Microsoft plans huge deployment of its own AI chips, with an eye on saving costs 

By Stephen Nellis and Max A. Cherney

SAN FRANCISCO, Oct 12 (Reuters) - Microsoft announced the next generation of its custom AI chip on Monday and said it plans to deploy potentially more than a million of them next year to back its internal AI services.

The Maia 300, as the chip is called, is the third generation of the company's custom chips designed to handle AI data crunching such as coding and carrying out complex business tasks on a user's behalf. Its predecessor is already running Microsoft's in-house AI models, as well as models from OpenAI delivered on Microsoft's Azure cloud computing service.

The new version comes amid a flurry of custom chips from both cloud computing rivals and AI firms such as OpenAI aimed at loosening the hold of Nvidia on the AI computing market.

To construct the chip, Microsoft took a different technical path from Nvidia whose AI accelerators use one massive computing chip surrounded by high-bandwidth memory chips.

The central computing processor in Microsoft's Maia 300 silicon is composed of "chiplets"  —   typically specialized chips bonded together  —  that use 2-nanometer technology from Taiwan Semiconductor Manufacturing Co for key computing functions.

Other chiplets included in the design use cheaper 3-nanometer tech for functions such as moving data off the processor and elsewhere inside the system.

The Maia 300 processors will go into racks that contain 96 chips and use about 167 kilowatts of power, compared with the 142 kilowatts used by Nvidia's most recent GB300 AI server racks with 72 of its graphics processing units (GPUs).

Microsoft said the Maia 300 chips will get about 40% more useful computing done per megawatt than "leading GPU-based systems," though it did not name the systems it used for a comparison.

"It's basically delivering more intelligence for every watt, every dollar, and every byte of memory by optimizing across the entire system," Rani Borkar, president of Azure hardware systems and infrastructure at Microsoft, told Reuters in an interview.

Microsoft said it plans to roll out "multi-gigawatt deployments" of the Maia 300 chip starting next year. Microsoft has said it expects to spend more than $175 billion in capital expenditures this calendar year.

(Reporting by Stephen Nellis and Max Cherney in San Francisco; Editing by Sanjeev Miglani)