Intel delivers ‘Real-Time AI’ in Microsoft’s new accelerated deep learning platform

Artificial intelligence (AI) and deep learning (DL) promise to transform how people engage with the world and how businesses operate in order to make smarter decisions, and Intel® technology is enabling this transformation. Today, Intel is highlighting how its AI technology is being leveraged by one of the world’s largest cloud service providers to run complex deep learning models, which enable applications ranging from facial recognition and voice recognition on smartphones to autonomous driving.

Microsoft has selected Intel® Stratix® 10 FPGAs as a key hardware accelerator in its new accelerated deep learning platform – code-named Project Brainwave. This FPGA-based accelerated deep learning platform is capable of delivering “real-time AI,” which will allow cloud infrastructure to process and transmit data as fast as it comes in, with ultralow latency. In the cloud, delivering real-time AI is becoming more important as systems are required to process live data streams, including video, sensors or search queries, and rapidly deliver the data back to users.

^{Stratix 10 FPGAs and SoC FPGAs leverage Intel’s 14nm process}

Through a blog issued today, Microsoft detailed how Project Brainwave, leveraging Intel FPGAs, is capable of handling extremely challenging deep learning models at “unprecedented” levels of performance and flexibility. Microsoft demonstrated its FPGA-based deep learning platform at Hot Chips 2017, a symposium that showcases the latest advancements in semiconductor technology.

Microsoft was the first major cloud service provider to deploy FPGAs in its public cloud infrastructure and the technology advancements it is demonstrating today with Intel Stratix 10 FPGAs enable the acceleration of deep neural networks (DNNs) that replicate “thinking” in a manner that is conceptually similar to that of the human brain.

AI is a rapidly evolving field that requires multiple technologies to efficiently manage various workload requirements. Intel offers a broad set of technologies to enable the market’s evolution, including Intel Xeon® processors, Intel FPGAs and Intel Nervana™ ASIC technology. Compared to dedicated deep learning hardware accelerators that are optimized to run a single workload, the flexibility of Intel FPGAs enable users to customize the hardware to meet specific workload requirements, and reconfigure the hardware rapidly as deep learning workloads and use models change. Intel Stratix 10 FPGAs combine hardened processor blocks that deliver high levels of sustained performance and efficiency, with a programmable fabric for user customization.

Many silicon AI accelerators today require grouping multiple requests together (called “batching”) to achieve high performance. Project Brainwave, leveraging the Intel Stratix 10 technology, demonstrated over 39 Teraflops of achieved performance on a single request, setting a new standard in the cloud for real-time AI computation. Stratix 10 FPGAs sets a new level of cloud performance for real-time AI computation, with record low latency, record performance and batch-free execution of AI requests.

“We exploit the flexibility of Intel FPGAs to incorporate new innovations rapidly, while offering performance comparable to, or greater than, many ASIC-based deep learning processing units,” said Doug Burger, distinguished engineer at Microsoft Research NExT.

Microsoft is currently working to deploy Project Brainwave in the Azure cloud so that customers eventually can run complex deep learning models at record-setting performance.

Machine learning platforms powered by Intel technology help transform data into actionable business intelligence through accelerated model training, fast scoring and highly scalable infrastructure. In conjunction with the latest Intel Xeon Scalable processors, Intel FPGAs are customizable and programmable to deliver low latency and flexible precision, with higher performance per watt for deep learning inference compared with Intel Xeon processors alone.