Topics
Latest
AI
Amazon
Image Credits:Fink Avenue(opens in a new window)/ Getty Images
Apps
Biotech & Health
Climate
Image Credits:Fink Avenue(opens in a new window)/ Getty Images
Cloud Computing
Commerce
Crypto
Image Credits:Microsoft
go-ahead
EVs
Fintech
Image Credits:Microsoft
fund-raise
Gadgets
Gaming
Image Credits:Microsoft
Government & Policy
Hardware
Image Credits:Microsoft
Layoffs
Media & Entertainment
Meta
Microsoft
concealment
Robotics
security system
societal
blank space
inauguration
TikTok
Transportation
Venture
More from TechCrunch
effect
Startup Battlefield
StrictlyVC
newssheet
Podcasts
video
Partner Content
TechCrunch Brand Studio
Crunchboard
Contact Us
Most company develop AI models , specially reproductive AI models likeChatGPT , GPT-4 TurboandStable Diffusion , trust hard on GPUs . GPUs ’ power to perform many computations in parallel make them well - suited to training — and running — today ’s most capable AI .
But there simply are n’t enough GPUs to go around .
Nvidia ’s best - performing AI cards arereportedlysold out until 2024 . The CEO of chipmaker TSMC was less optimistic lately , suggestingthat the shortage of AI GPUs from Nvidia — as well as chip from Nvidia ’s rivals — could poke out into 2025 .
So Microsoft ’s going its own means .
Today at its 2023 Ignite conference , Microsoft unveil two custom - designed , in - house and data point plaza - bound AI chips : the Azure Maia 100 AI Accelerator and the Azure Cobalt 100 CPU . Maia 100 can be used to develop and run AI models , while Cobalt 100 is designed to consort universal purpose workloads .
“ Microsoft is building the substructure to stand AI innovation , and we are reimagining every aspect of our information centers to meet the motivation of our client , ” Scott Guthrie , Microsoft cloud and AI group EVP , was cite as enounce in a jam spill provide to TechCrunch in the first place this week . “ At the weighing machine we operate , it ’s crucial for us to optimise and desegregate every layer of the infrastructure push-down stack to maximize operation , diversify our supply chain and give client infrastructure choice . ”
Both Maia 100 and Cobalt 100 will begin to roll out too soon next class to Azure data centers , Microsoft say — initially power Microsoft AI service likeCopilot , Microsoft ’s family of productive AI ware , andAzure OpenAI Service , the company ’s fully managed offering forOpenAImodels . It might be early days , but Microsoft assures that the micro chip are n’t one - offs . Second - generation Maia and Cobalt hardware is already in the work .
Join us at TechCrunch Sessions: AI
Exhibit at TechCrunch Sessions: AI
Built from the ground up
That Microsoft created custom AI french fries does n’t come as a surprise , exactly . The wheel were set in motion some clip ago — and publicize .
In April , The Informationreportedthat Microsoft had been working on AI french fries in secret since 2019 as part of a projection code - diagnose Athena . And further back , in 2020 , Bloombergrevealedthat Microsoft had project a mountain chain of chip base on the ARM computer architecture for data eye and other devices , including consumer ironware ( suppose theSurface Pro ) .
But the announcement at Ignite gives the most thorough smell yet at Microsoft ’s semiconductor gadget travail .
First up is Maia 100 .
Microsoft says that Maia 100 — a 5 - nanometer chip contain 105 billion transistors — was mastermind “ specifically for the Azure hardware stack ” and to “ achieve the absolute maximum utilization of the hardware . ” The company promises that Maia 100 will “ power some of the largest inner AI [ and productive AI ] workloads run on Microsoft Azure , ” inclusive of workloads for Bing , Microsoft 365 and Azure OpenAI Service ( but not public swarm customer — yet ) .
That ’s a destiny of jargon , though . What ’s it all mean ? Well , to be quite true , it ’s not wholly obvious to this newsperson — at least not from the details Microsoft ’s allow in its insistence materials . In fact , it ’s not even unclouded what kind of chip Maia 100is;Microsoft ’s chosen to keep the architecture under wraps , at least for the prison term being .
In another disappointing development , Microsoft did n’t subject Maia 100 to public benchmarking run suite likeMLCommons , so there ’s no liken the chip ’s execution to that of other AI training chips out there , such as Google’sTPU , Amazon’sTraniumand Meta’sMTIA . Now that the cat ’s out of the bag , here ’s hope that ’ll change in scant ordering .
One interesting factoid that Microsoftwaswilling to discloseis that its close AI partner andinvestment target , OpenAI , provided feedback on Maia 100 ’s figure .
It ’s an evolution of the two company ’ compute infrastructure draw - ups .
In 2020 , OpenAIworkedwith Microsoft to co - contrive an Azure - host “ AI supercomputer ” — a clump hold over 285,000 processor cores and 10,000 graphics cards . afterward , OpenAI and Microsoft built multiple supercomputing arrangement powered by Azure — which OpenAI exclusively uses for its research , API and products — to condition OpenAI ’s models .
“ Since first partnering with Microsoft , we ’ve collaborated to co - project Azure ’s AI infrastructure at every layer for our models and unprecedented preparation needs , ” Altman said in a canned statement . “ We were excited when Microsoft first shared their designs for the Maia chip , and we ’ve worked together to refine and test it with our model . Azure ’s death - to - end AI computer architecture , now optimized down to the silicon with Maia , paves the way for train more subject models and give those models cheap for our client . ”
I expect Microsoft for clarification , and a spokesperson had this to say : “ As OpenAI ’s exclusive cloud provider , we forge intimately together to secure our infrastructure meets their requirements today and in the future . They have provided valuable testing and feedback on Maia , and we will continue to consult their roadmap in the development of our Microsoft first - company AI silicon generations . ”
We also know that Maia 100 ’s strong-arm package is larger than a distinctive GPU ’s .
Microsoft says that it had to build from scratch the data nerve center host racks that house Maia 100 chips , with the goal of accommodate both the chips and the necessary power and networking cables . Maia 100 also involve a singular liquidness - base chilling solution since the chip shot consume a high - than - average amount of power and Microsoft ’s data centers were n’t contrive for orotund liquid chillers .
“ Cold liquid flows from [ a ‘ sidekick ’ ] to insensate photographic plate that are bind to the surface of Maia 100 chip , ” explains a Microsoft - author post . “ Each home plate has groove through which liquid is circulated to occupy and transferral heat . That flow to the sidekick , which removes passion from the liquid state and post it back to the single-foot to absorb more passion , and so on . ”
As with Maia 100 , Microsoft kept most of Cobalt 100 ’s technical detail vague in its Ignite unveiling , deliver that Cobalt 100 ’s an energy - effective , 128 - core chip built on anArm Neoverse CSSarchitecture and “ optimise to have big efficiency and execution in cloud native offerings . ”
subdivision - based AI inference chips were something of a drift — a trend that Microsoft ’s now perpetuating . Amazon ’s latest datum nitty-gritty silicon chip for illation , Graviton3E(which complementsInferentia , the company ’s other inference chip shot ) , is build on an Arm architecture . Google isreportedlypreparing customs Arm host chips of its own , meanwhile .
“ The architecture and implementation is project with power efficiency in mind , ” Wes McCullough , CVP of hardware mathematical product development , said of Cobalt in a statement . “ We ’re build the most efficient consumption of the transistor on the silicon . Multiply those efficiency addition in servers across all our datacenters , it sum up up to a passably big number . ”
A Microsoft voice say that Cobalt 100 will power unexampled practical machines for customer in the forthcoming year .
But why?
So Microsoft ’s made AI buffalo chip . But why ? What ’s the need ?
Well , there ’s the company line — “ optimize every layer of [ the Azure ] engineering muckle , ” one of the Microsoft blog posts published today reads . But the subtext is , Microsoft ’s vie to stay militant — and cost - conscious — in the relentless race for AI dominance .
The scarceness and indispensability of GPUs has left company in the AI blank space large and small , including Microsoft , beholden to chip vendors . In May , Nvidiareacheda marketplace value of more than $ 1 trillion on AI scrap and related gross ( $ 13.5 billion in itsmost late fiscal quarter ) , becoming only the sixth tech company in account to do so . Even with a fraction of the install nucleotide , Nvidia ’s chief rival , AMD , expectsits GPU data center revenue alone to eclipse $ 2 billion in 2024 .
Microsoft is no doubt dissatisfied with this arrangement . OpenAI sure as shooting is — and it ’s OpenAI ’s tech that drives many of Microsoft ’s flagship AI products , apps and service today .
In aprivate meetingwith developers this summer , Altman admitted that GPU famine and costs were hindering OpenAI ’s progress ; the fellowship just this week wasforcedto pause augury - ups for ChatGPT due to capacitance issues . emphasise the degree , Altman said in aninterviewthis week with the Financial Times that he “ hoped ” Microsoft , which has gift over $ 10 billion in OpenAI over the past four years , would increase its investing to help pay for “ huge ” impending poser education cost .
Microsoft itselfwarnedshareholders earlier this yr of potential Azure AI service disruptions if it ca n’t get enough chips for its data centers . The company ’s been storm to take drastic measure in the meanwhile , like incentivizing Azure customers with unused GPU reservation to give up those reservations in exchange for refund and pledgingupwardsof billion of dollar to third - party swarm GPU providers likeCoreWeave .
Should OpenAIdesignits own AI chip as rumor , it could put the two party at odds . But Microsoft in all likelihood sees the potential monetary value savings arise from in - house hardware — and competitiveness in the cloud market — as deserving the risk of preempt its friend .
One of Microsoft ’s premier AI products , the code - generatingGitHub Copilot , has reportedly been costing the society up to $ 80 per drug user per month partially due to model inferencing costs . If the position does n’t call on around , investment house UBSseesMicrosoft struggling to sire AI tax income streams next class .
Of course , ironware is heavy , and there ’s no guaranty that Microsoft will succeed in launching AI chips where others fail .
Meta ’s former custom AI chip efforts were beset with problems , contribute the caller to scrap some of its experimental hardware . Elsewhere , Google has n’t been capable to keep footstep with need for its TPUs , Wiredreports — and run into designissueswith its newest coevals of the microchip .
Microsoft ’s chip in it the old college try , though . And it ’s oozing with confidence .
“ Microsoft innovation is going further down in the stack with this atomic number 14 piece of work to ensure the time to come of our customers ’ workload on Azure , prioritizing public presentation , exponent efficiency and cost , ” Pat Stemen , a partner program manager on Microsoft ’s sky-blue hardware systems and base squad , say in a blog post today . “ We chose this innovation deliberately so that our customers are going to get the best experience they can have with Azure today and in the futurity … We ’re trying to provide the good solidification of options for [ client ] , whether it ’s for carrying into action or cost or any other property they care about . ”
