Dell Offers AMD’s Cutting-Edge Instinct MI300X AI Accelerators In “PowerEdge” AI Servers

AMD's Instinct MI300X AI Throughput Performance & Latency Improved By 7x With GEMM Tuning 1

Dell has extended its cooperation with AMD, now offering Team Red's highest-end Instinct MI300X AI accelerators in its cutting-edge servers along with new generative AI solutions.

[Press Release]: As AI continues to rapidly evolve, the fusion of open-source technologies and cutting-edge hardware acceleration is driving industry innovation. Dell Technologies and AMD are at the forefront, offering on-premises infrastructure solutions that are up to 75% more cost-effective than public cloud IaaS and tailored to empower generative AI applications in the enterprise.

We’re excited to share a series of updates powering the deployment of open AI ecosystems and frameworks. First, the Dell PowerEdge XE9680 is now shipping with the AMD Instinct MI300X accelerators, enhancing AI performance options for our customers. Second, we’ve published the Dell Validated Design for Generative AI with AMD, enabling custom application development with open-framework foundations. Third, our new deployment services, available in September, will ensure the smooth integration of these innovations into your operations.

The PowerEdge XE9680 with AMD Instinct MI300X accelerators offers high-performance capabilities designed for enterprises leveraging generative AI and features eight MI300X accelerators, a combined 1.5 TB of HBM3 memory, and 42 petaFLOPS of peak theoretical FP8 with sparsity precision performance.

This powerful configuration enables faster training and inferencing of large language models helping organizations deliver AI-driven insights and innovative applications more efficiently. Recent testing on a single node configuration displayed a leading total cost of ownership value:

Deployed the Llama 2 70B parameter model on a single AMD Instinct MI300X accelerator on the Dell PowerEdge XE9680 Server.

Deployed eight concurrent instances of the model by utilizing all eight available AMD Instinct MI300X accelerators on the Dell PowerEdge XE9680 Server.

Fine-tuned the Llama 2 70B parameter model with FP16 precision on one Dell PowerEdge XE9680 Server with eight AMD Instinct MI300X accelerators.

With simplified deployment through Dell OpenManage Enterprise, intelligent automation via APEX AIOps software, and enhanced security featuring integrated cyber recovery and a Zero Trust approach, the XE9680 empowers businesses to rapidly implement and scale their GenAI solutions while maintaining a robust security posture.

Announced in May, and available today, the Dell Validated Design for Generative AI with AMD is the next step of Dell Generative AI Solutions making it easier for organizations to deploy trustworthy GenAI. This design guidance gives organizations and developers comprehensive directions to implement LLM inferencing and model customization, as well as advanced techniques like fine-tuning and retrieval augmented generation (RAG). Built on open standards and reducing the need for proprietary AI software suites, developers can simplify development and freely customize workflows with open-source LLM models from partners including Hugging Face and Meta.

Accelerate modern workloads. Innovation and scale that enable efficient, agile businesses and outcomes.

Powered by AMD Instinct MI300X accelerators, the Dell Validated Design enables near-linear scaling and low latency distributed GenAI training and inferencing.

The PowerScale F710 delivers faster time-to-air insights with massive gains in streaming performance that accelerate all phases of the AI pipeline.

The Dell PowerSwitch Z9664F-ON, offering 64 ports of 400GbE, delivers low latency and high throughput Ethernet fabrics for modern AI clusters.

The Broadcom Thor2 AI-optimized NIC delivers 400G, interconnecting MI300X accelerators with the industry’s lowest power requirements.

Boost application development. Open-source software and ecosystems allow developers and data scientists to innovate freely.

AMD ROCm-powered frameworks extend the Dell Generative AI Solutions ecosystem and include support for open-source large language models like PyTorch, TensorFlow, ONNX-RT, and JAX, as well as the full stack of drivers, dev toolkits and APIs for AMD Instinct accelerators.

Dell Omnia streamlines the creation and management of AI clusters automating configuration for efficient workload processing.

Enterprise SONiC distribution by Dell Technologies delivers a scalable networking solution that combines the open-source SONiC platform with Dell PowerSwitch, offering advanced features and enterprise-grade support.

Dell Validated Designs for Generative AI make it simple for Dell customers to build GenAI platforms tailored to their needs by taking the guesswork out of integration, performance, and sizing considerations.

News Source: Dell