MS Ignite 2023 (Silicon edition)
Posting about today's #microsoftignite2023 announcement in Seattle, from #sc23 in Denver, while finishing an article on #ocpsummit23 in San Jose is frankly...a little meta - but here we go.
Whilst Microsoft cover a lot of software updates in the keynote, I'm going to cover the silicon announcements, their new CPU (Cobalt 100) and AI (Maia 100) accelerator.
Cobalt is an #arm based processor with 128-cores (similar to an #ampere processor)
Maia is a 5nm ASIC (not GPU) with 105 billion transistors (compared with 80 billion in a #nvidia H100)
My initial thoughts... 🤔
Cobalt
Performance comparisons were very vague (40% better was used) however Arm use the same "40% better" claim when comparing current gen N2 architecture to N1. What is interesting, is that it is running internal production workloads today.
https://www.arm.com/products/silicon-ip-cpu/neoverse/neoverse-n2
Maia
Most interesting (to me at least), is Maia.
Whilst no real specs were provided, Microsoft did show that they've had to develop an in-house liquid cooling solution to manage it, which gives an idea about how much power it pulls. Liquid cooling has absolutely gone mainstream, something I confirmed in person this week at #sc23. This will allow Microsoft to utilize their existing data center infrastructure, after some plumbing upgrades, to serve this infrastructure, albeit at 2x the tile footprint per rack.
Also of note, this is an ASIC, not a GPU, so visual computing will continue to require an NVIDIA or AMD (or Intel) GPU.
I would expect Maia to be primarily used for broad internal inferencing use (think Co-Pilot) not LLM training, and as a 3rd (perhaps cheaper) option for Azure customers where supply is limited. I'm very much looking forward to seeing performance figures.
Summary
The word partnership at Microsoft is doing some heavy lifting, but just as other hyperscaler's produce their own silicon, Microsoft are doing the same. Curiously, Maia was announced after first announcing both NVIDIA's new H200, and AMD's new MI300X becoming available on Azure, and 10mins before Jensen surprises the audience on stage to discuss NVIDIA's software services on Azure.
I'm very curious about actual specs, and real-world performance and benchmarks of both chips, as only then will comparisons be possible. Also, how this might work in a world where CUDA is being used. The software implications might be challenging.
Until then we get to ponder wonder what a Cobalt and H200 solution would perform like (and cost) compared to a GH200 solution, and the traditional x86 CPU solutions we have seen to date.
2024 is going to be wild.