Four takeaways from Intel’s Investor Webinar

All eyes — especially investors’ eyes — are on Intel’s datacentre business today.

Intel’s Sandra Rivera, Greg Lavender and Lisa Spelman hosted a webinar focused on the company’s Data Center and Artificial Intelligence business unit. They offered a big update on Intel’s latest market forecasts, hardware plans and the way Intel is empowering developers with software.

Executives dished out updates on Intel’s datacentre business for investors. This included disclosures about future generations of Intel® Xeon® chips, progress updates on 4th Gen Intel® Xeon® Scalable processors (code-named Sapphire Rapids) and demos of Intel hardware tackling the competition, heavy AI workloads and more.

Xeon Roadmap Roll Call

Among Sapphire Rapids, Emerald Rapids, Sierra Forest and Granite Rapids, there is a lot going on in the server CPU business. Here’s your Xeon roadmap updates in order of appearance:

Today: A 4th Gen Intel Xeon Scalable update

All major original equipment manufacturers (OEMs) and original design manufacturers (ODMs) are shipping 4th Gen Xeon systems, and the top 10 global cloud service providers are working on deploying services on 4th Gen Xeon. It’s the highest quality data center CPU Intel has ever delivered, and the ramp is continuing aggressively. Lisa Spelman, corporate vice president and general manager of Intel Xeon Products, offered up a demo of a 48-core 4th Gen Xeon going head-to-head with a 48-core 4th Gen AMD Epyc CPU. The result? Intel’s 4th Gen Xeon with Intel® Advanced Matrix Extensions (Intel® AMX) delivers an average performance gain of 4 times the competition’s latest tech on a broad set of deep-learning workloads.² 

Q4 2023: 5th Gen Intel Xeon Scalable (code-named Emerald Rapids)

Emerald Rapids is Intel’s next Performance-core (P-core) product. Starting today, it is officially known as 5th Generation Intel® Xeon® Scalable. As it sheds its code name and dons formal branding in preparation for its launch, the CPU is already sampling to customers. Volume validation is underway and Sandra Rivera, executive vice president and general manager of the Data Center and AI Group, told investors that factory silicon quality is very high.

First-half 2024: Intel Xeon Processor code-named Sierra Forest

Intel® Xeon® Processor code-named Sierra Forest, the first Efficient-core (E-core) Xeon processor, is on track for the first half of 2024. Today, Intel announced that Sierra Forest will boast 144 cores per socket. The first CPU based on the upcoming Intel 3 process, Sierra Forest hit power-on earlier this quarter with multiple OSes booting in record time (less than a day). It’s on schedule with the first samples already out the door. A Sierra Forest demonstration during today’s presentation included a nice shot of task manager showing all 144 cores chugging along processing the demo workload.

A Fast Follow: Intel Xeon Processor code-named Granite Rapids

Intel® Xeon® Processor code-named Granite Rapids will arrive hot on the heels of Sierra Forest in 2024. Though a specific launch date has yet to be disclosed, its time-to-market will benefit from sharing a platform with Sierra Forest. Shared IP and technology minimize development and design time. Granite Rapids is hitting all major milestones and the first stepping is out of the factory and healthy. It’s also sampling to customers with positive feedback. Spelman hosted a first taste of Granite Rapids innovation during a demo.

“We are building the fastest memory interface in the world for Granite Rapids,” Spelman said as she fired up a command prompt to show off its memory configuration. “Intel invented and led the ecosystem in developing a new type of DIMM called Multiplexer Combined Rank (MCR) that lets us achieve speeds of 8,800 mega transfers per second, based on DDR5.”

The MCR DIMM innovation achieves an 80% peak bandwidth increase over current-gen server memory technology, and Lisa was able to demonstrate Granite Rapids’ stability while saturating a healthy memory subsystem with read/writes.

Further into the Future

Speaking to it publicly for the first time, Intel will continue to execute on its E-core roadmap with the follow-on to Sierra Forest: Clearwater Forest. Coming to market in 2025, Clearwater Forest will be manufactured on Intel 18A, the node where Intel plans to achieve process leadership – it's the culmination of the company’s five-nodes-in-four-years strategy.

Bonus Bits

• The Habana® Gaudi®3 AI accelerator has taped in. Performance details haven’t been shared yet, but Habana® Gaudi®2 is in the environment and offering 1.8x advantage in throughput-per-watt over a comparable A100 server when running a popular computer vision workload.¹
• There are 15 new FPGA products scheduled to go through the production release qualification (PRQ) process this year – that’s more new product introductions than ever before in Intel’s FPGA business.

Looking Beyond the CPU for $110 Billion

This packed roadmap is on track to deliver against an even bigger total available market (TAM) than previously considered. The $110 billion five-year TAM that Rivera cited for the data center silicon business is roughly double the opportunity highlighted during last year’s Intel Investor Day. The reason? A changing data center landscape that goes beyond the CPU.

“When we talk about compute demand, we often look at the TAM through the lens of CPU units,” Rivera explained. “However, counting sockets does not fully reflect how silicon innovations deliver value to the market. Today, innovations are delivered in several ways, including increased CPU core density, the use of accelerators built into the silicon and the use of discrete accelerators.”

Based on the above, Rivera cited a couple of specific reasons for the increased TAM: With the integration of accelerator compute and advanced GPU offerings into its data center business, Intel is better positioned to serve a wider swath of customers. What’s more, Xeon’s performance in high-profile workloads (like AI, analytics, security, networking and HPC) is driving demand for mainstream compute as well as discrete accelerators tailored for the task at hand.

AI is Everywhere, It’s for Everyone and It’s Running on Intel

Silicon updates aside, AI was a persistent theme in today’s webinar.

Intel is already foundational in AI hardware, from data preparation and management to small- and medium-scale training and inference on Xeon, and increasingly large model training and inference using Intel GPUs and accelerators.

Today, Rivera defined Intel’s commitment to the true democratization of AI from the cloud to the network and to the edge by enabling broader access to solutions and more cost-effective deployments through an open ecosystem approach.

“Customers want portability in their AI workloads. They want to build once and deploy anywhere,” Rivera said. “As we continue to deliver heterogenous architectures for AI workloads, deploying them at scale will require software that makes it easy for developers to program and a vibrant open and secure ecosystem to flourish.”

Greg Lavender, chief technology officer and leader of Intel’s Software and Technology Group, was on hand to talk about Intel’s advanced software as well as its investment in a holistic, end-to-end systems-level approach to AI software – including work to standardize programming languages for the portability Rivera mentioned.

“The desire for an open, multivendor multiarchitecture alternative to (Nvidia’s) CUDA is not waning,” Lavender said. “We believe that the industry will benefit from a standardized programming language that everyone can contribute to, collaborate on, is not locked into a particular vendor and can evolve organically based on its members and public requirements.”

Intel has made contributions to SYCL, an open C++-based programming model, and acquired Codeplay Software (a leader in the SYCL language and community). SYCL is now included in oneAPI so customers can program and compile across CPUs, GPUs and accelerators from multiple vendors. Additional work on software optimizations upstream includes optimizations for PyTorch 2.0 and TensorFlow 2.9, as well as a collaboration with Hugging Face to train, tune and predict with the help of Intel Xeon and Gaudi 2.

Powering Progress in Generative AI

Intel’s open philosophy extends to the field of generative AI, where powerful tools like ChatGPT and text-to-image model DALL·E 2 are leveraging AI algorithms and large data sets to produce new content based on increasingly complex human prompts.

Generative AI is evolving at warp speed (it feels like there are near-daily headlines about breakthroughs), and Intel is actively working to support an open generative AI ecosystem while also addressing increasing performance demands.

Earlier this week, Hugging Face, a major player in machine learning application development, announced that it had enabled the 176 billion parameter BLOOMZ model on Habana’s Gaudi 2. The BLOOM model is an open source large language AI model analogous to the 175B parameter GPT-3 model employed by ChatGPT. The company also confirmed it is running deep-learning text-to-image model Stable Diffusion on 4th Gen Xeon with built-in Intel AMX for AI workload acceleration (Stable Diffusion is an open access alternative to DALL·E).

Beyond Intel’s efforts to bolster an open and ethical ecosystem, Xeon is present and accounted for elsewhere in generative AI landscape, too. Rivera reminded investors today that Nvidia is using 4th Gen Xeon as the head-node to run alongside its H100 GPUs to power the virtual machines accelerating generative AI models in Microsoft Azure, including ChatGPT.

¹ _{Supermicro L12 Validation Report of Gaudi2 HL-225H SYS-820GH-THR2, Oct. 20, 2022.}

² _{4th Generation Intel® Xeon® Scalable Processors - 1 | Performance Index}