Intel & BCG Join Hands To Boost Generative AI With Habana Gaudi Accelerators

Intel and BCG have joined hands as part of their new strategic partnership to boost Generative AI with Habana Labs Gaudi accelerators.

Generative AI and AI, in general, is the hot topic these days. NVIDIA is currently leading this space with its AI GPUs and AMD is following suit. Now, Intel has also decided to enter the segment in a brand new partnership with BCG who want to deliver enterprise-grade and secure Generative AI.

For this purpose, BCG will be leveraging Intel's hardware ecosystem which includes Xeon Scalable Processors and AI-optimized Habana Gaudi accelerators along with a range of hybrid cloud-scale software.

“Generative AI is an emerging and dynamic space, which means organizations must pick the right technology to power their GenAI journey,” says Suchi Srinivasan, a managing director and partner at BCG. “The technology must be enterprise grade from day one and allow for privacy, security, ease of use, and scalability. Our collaboration with Intel will enable enterprises to develop competitive advantages via custom GenAI solutions while thoughtfully navigating the people, process, and policy changes required to derive the maximum value from these transformative solutions.

Generative AI requires a truly democratized approach that enables more secure and scalable choice so enterprises can safely benefit from the technology,” said Sandra Rivera, executive vice president and general manager of the Data Center and AI Group at Intel. “Our collaboration with BCG allows us to help customers build generative AI applications that require technology optimized across the entire stack completely inside their chosen security perimeter.”

via Intel / BCG

Intel is currently offering its 2nd Gen Habana Gaudi accelerators based on the 7nm process node, offering 24 TPCs for media decode and processing running on an FP8 format (versus 8 TPCs). The memory configuration includes 96 GB of HBM2e memory, offering 2.45 TB/s bandwidth and an additional 48 MB of SRAM. Networking is provided through 24 100GbE switches. Such a big jump in performance also means that the TDP has to be upped dramatically & the Gaudi2 operates at a 600W TDP (versus 350W). These chips are said to be competitive against NVIDIA's A100 AI GPUs in performance/cost.

The company is also working on its 3rd Gen Habana Gaudi accelerators which will utilize the TSMC 5nm process node and deliver even higher performance and efficiency boost. Compared to NVIDIA H100, the latest offering from the green team, the Gaudi3 accelerator will be very competitive in price to performance. There are even going to be hybrid solutions that combine the tech expertise of Haban Gaudi accelerators (4th Gen) with Intel's hyper-scale GPUs such as Ponte Vecchio from AXG in the future.

“Gaudi3 is coming really soon,” Medina said. “It’s actually in manufacturing. It is going to be our TSMC 5nm product.

Right now, Ponte Vecchio is focused on Argonne National Lab, right? More of the HPC use case,” Medina said. “Intel is recognizing that if that server needs to do just AI – if it’s a heavy load – then it’s Gaudi2, and it will be Gaudi3 immediately after that. Now, the next generation – the fourth generation – is going to combine the Gaudi capabilities and some of the AXG capabilities.

We’re already in design of what will be the next-generation accelerator,” Medina added. “Intel will be actually, really unifying the roadmaps between Habana and the AXG side of the organization. We are working on a deeper integration.”

Intel Habana COO Eitan Medina (via HPC Wire)

It definitely looks like Intel won't be waiting anymore as others climb the AI ladder & will be joining the race too. Once again, the others, especially NVIDIA, have years worth of lead in the AI segment so it will be some catching up to do for the other players.