Intel Talks AI Strategy: Next-Gen Gaudi 3 & Falcon Shore Accelerators, 3rd Party Migration, China SKUs

Intel shared a few more updates on its AI strategy and accelerators including next-gen Gaudi 3 and Falcon Shores which reveal how the company is bringing AI to the enterprise and all aspects of the data center segment with its products and software stack.

For AI, Intel has developed a full-on Enterprise AI stack, a combination of hardware and AI Applications/Software developed using open standards that are possible with OpenVINO & the Intel Developer Cloud oneAPI & Synapse. The hardware is segmented into three branches, data center (Scalable Systems, Accelerators, CPUs), Networking (Open Standards and Configurability, Infrastructure), and Client & Edge (AI PC, NPU, GPU, CPU).

NVIDIA has long been the sole provider of high-performance and capable AI accelerators in the market but that has changed with Intel and AMD along with others prepping up new solutions & they look very performant (on paper for now). Intel wants to establish itself as a clear alternative to the market leader and at the same time, they are going to establish leadership in specific markets and workloads. We recently saw benchmarks from Stability AI showcasing Intel's Gaudi 2 hardware offering a 3x boost at a similar price.

Intel states that in terms of pricing, Gaudi 2 is based on the same process as the A100 which is TSMC's 7nm but the key difference is that their customers are seeing up to 3x the performance improvement in Stability Diffusion and Generation AI workloads which means that you can get a lot more cost savings with using Gaudi 2 than NVIDIA's A100. The Gaudi 2 hardware also excels over the latest H100 GPU in certain workloads which is exactly what Intel is gunning for now and will scale the leadership beyond from here.

Coming to the roadmap, Intel still has Gaudi 3 positioned as the follow-up to Gaudi 2 and Falcon Shores beyond that. Intel stated that Gaudi 3 accelerators are already in its validation labs right now and will be generally available in Q3 2023.

Gaudi 3 will also be available through a broad ecosystem of OEMs and also in the Intel developer cloud. In terms of hardware, the Gaudi 3 AI accelerator will utilize the same high-performance architecture as Gaudi 2 with significant advances in computing capabilities. You are getting 4x the compute, 2x the networking bandwidth, 1.5x the HBM memory bandwidth, and higher capacities to allow for larger models.

The Falcon Shores will be the first GPU architecture designed solely for AI workloads. It combines the best of the Gaudi AI IP as well as Intel GPU technology into a converged product offering. The development of Falcon Shores is well underway and is aiming for a 2025 launch.

One major concern for AI developers is that migrating to new hardware comes with its issues such as modifying the code to enable support for the new accelerators. According to Intel, most developers are working on the high level of the stack (Hugging Face, Mosiac ML, PyTorch, etc), and with Intel's software solutions they can leverage the fact that with just a few lines of code to their Python script, they can migrate their existing models or new ones.

Intel's Model Migration requires just 3-5 lines of code to migrate existing or new models to the new hardware. For high-level software, Intel offers migration support from NVIDIA to its entire Gaudi & even the next-gen Falcon Shores accelerator lineup. The company is also working on adding low-level models such as OpenAPI, Triton, and TPC-C to be supported on Falcon Shores GPUs when moving from NVIDIA hardware.

Talking about its China plans, Intel said that they fully comply with the regulations from US authorities and that new variants based on the Gaudi hardware are being adopted to make sure that they are exportable to China.

We are focusing on making sure that we fully comply with the export requirements. We've always done that and we are very very close with the regulator that whatever we offer to the PRC customers fully comply with the regulations. You have seen the announcement of a Gaudi 2 SKU last year and as the regulation change, we also the definition of the SKUs that we can provide to the PRC market. We have important customers in China and we are making sure to follow the regulations and we do expect to introduce SKUs that meet the regulations using the same Gaudi hardware but with adoptions that make them exportable to China.

via Intel

When asked about whether Intel was going after training and inference segments, the company stated that they were aiming for both. Generative AI and LLM are the ones driving demand. NVIDIA in its investor call last year also stated that the AI demand coming from GenAI is going to be huge and that applies to all hardware vendors including Intel. Intel expects that the inference demand will increase over time but they are focused on providing a complete package that excels at both training and inference tasks.