Intel Arc GPUs Now Supported In The Intel Extension For PyTorch, Boosting AI, Deep-Learning & LLM Capabilities

Intel Arc GPUs Now Supported In The Intel Extension For PyTorch, Boosting AI, Deep-Learning & LLM Capabilities 1

Arc A-Series GPUs are now supported within the Intel Extension for PyTorch (IPEX), offering faster AI capabilities in deep learning & LLMs.

Intel's Arc A-Series GPUs based on the Alchemist architecture have loads of on-hardware potential that are getting realized day after day. The software division has done a great job in optimizing the game performance of these GPUs and now the focus is on the emerging AI market where the XMX units featured onboard these chips will get fully utilized.

In an update, Intel has confirmed that IPEX (Intel Extension for PyTorch) now offers support for Arc A-Series GPUs across Windows, Linux, and WSL2 platforms. With this newly added support, Intel's A Series GPUs would not only boost the acceleration of deep learning models such as LLMs but also bring new optimizations for Intel hardware, enabling faster performance & giving users the ability to run PyTorch models on the Alchemist line of GPUs.

For those of you who have an Intel Arc A-Series GPU and want to take advantage of their XMX AI engine within PyTorch, the company has posted a full blog on how to run the Llama 2 interface on Windows and WSL2 here. According to Intel's AI software team:

GPUs are well suited for large language model (LLM) workloads as GPUs excel at massive data parallelism and high memory bandwidth. The Arc A-Series Graphics, including Arc A770 Graphics, are high-performance graphics, with up to 512 Xe Matrix Extensions (XMX) Engines, providing capability for accelerated execution of deep learning models, including LLMs.

IPEX extends PyTorch with the latest performance optimizations for Intel hardware, taking advantage of XMX Engines on Intel discrete GPUs.

IPEX Extension for PyTorch enables PyTorch XPU device, which allows it to easily move PyTorch model and input data to the device to execute on Intel discrete GPU with GPU acceleration.

The latest release of IPEX (v2.1.10+xpu) officially supports Arc A-Series Graphics on WSL2, native Windows and native Linux.

Demonstrated running Llama 2 7B and Llama 2-Chat 7B inference on Arc A770 Graphics on Windows and WSL2 via Ipex.

To run Llama 2, or any other PyTorch models, on Arc A-Series GPUs, simply add a few additional lines of code to import intel_extension_for_pytorch and .to("xpu") to move model and data to device to execute on Arc A-Series GPU.

We recently posted an article quoting Intel's ex-GPU boss, Raja Koduri, who mentioned how client and gaming PC GPUs played a crucial role in enabling AI success. Intel expanding support to its Arc A-Series GPUs is a great thing for the market segment where NVIDIA and AMD have been leading the charge by offering swift AI support on consumer-centric hardware.