Data Center Strategy & Optimization Insights

AI for real life

As I’ve been busy with my day job(s) and various projects, like the Tech Insider Podcast, I haven’t put my hands to the keyboard for an article in a while. In this piece, while I’m still talking about AI, I want to demonstrate a few use cases that have significantly impacted my daily life. It’s easy to get caught up in the news cycle and hype about 100,000 GPUs being deployed, requiring nuclear power, and dealing with sustainability challenges—not to mention the cooling requirements now and in

To InfiniBand, maybe beyond?

Nvidia's latest roadmap was teased at Computex in Taiwan last month. Whilst details were a little light on PFLOPS and TDP for either the GPU or CPU, we did get some interesting information for the next-gen products. * GPU: Rubin (HBM3e to HBM4 memory) - TSMC 3N process * CPU: Vera (NVIDIA's 2nd gen ARM processor) - TSMC 3N process * Interconnect: NVLink6 (2x performance to 3600 GB/sec) * NIC: ConnectX9 (2x speed to 1.6Tb/sec) * Switch: SpectrumX1600 (2x speed to support CX9 NICs) NVIDIA

Apple, not Artificial, Intelligence

Just last month, Apple hosted their yearly WWDC - an event where they showcase all the updates to their platforms. Whilst a lot of it is very interesting, and AI centric, I'm going to mostly focus on Private Cloud Compute. But first…the first half. WWDC Regular Programming The first hour of the keynote provided great updates for the Apple ecosystem. I'm personally excited about Siri getting a huge kick in the pants, and into this decade, plus a bunch of quality-of-life upgrades across each

Oh great, another podcast...

As you may have seen (or heard my "Ausmerican" accent) recently, I've started a podcast, and wanted to share a little insight into why. I created this, "Infrastructure as a Newsletter", for *checks calendar* just shy of a year ago, and grateful to now have over 1300 subscribers. For something I was quite hesitant to start, I have definitely seen the fruits of my labor, and thankful my wife Victoria Hume encouraged me to put hands to keyboard, and to wade into the uncomfortable. I have quite en

OCP 2024 Regional Summit wrap

The Open Compute Project (OCP) Regional Summit was hosted in Lisbon, Portugal last month, the 5th (and largest) regional summit the group has hosted. Whilst I wasn't able to make it in person, I’d be remiss if I didn't write a (very) quick summary about the conference, and pertinent updates to scaling digital infrastructure in a sustainable way. The hot topic continues to be GenAI, such that OCP has created a new track for Artificial Intelligence, and a strategic initiative for Open AI Systems

Here come the Inferencing ASIC's

The tidal wave of Generative AI (GenAI) has mostly consisted of training large language models (LLM's), like GPT-4, and the huge amount of compute needed to process these enormous datasets, e.g. GPT-4 has 1.76 trillion parameters. This compute has mainly looked like NVIDIA's GPUs, but you also need... 1. power 2. networking 3. capital, AND 4. a nice cool place to host them (data center) The looooooong tail of AI Inferencing will dictate that compute is installed closer to where it's neede

GTC 2024 post-conference

Upon returning from GTC24, I've been able to reflect on all the new updates across NVIDIA's platforms and below is a summary of the various announcements. Blackwell was the star of the show, with the B100, B200 and GB200 chips announced. Note that there were no consumer facing graphics cards (RTX) were named, nor was there a successor to the L40S or BlueField3 DPU (though there was a new ConnectX8 NIC). As always though, a little bit of devil is in the details - as well as marketing waxing lyr

GTC 2024 preview