NVIDIA Hopper H200 GPUs Supercharged With World’s Fastest HBM3e Memory, Grace Hopper Superchips Power Jupiter Supercomputer

NVIDIA has announced its brand new H200 Hopper GPU which now comes equipped with the world's fastest HBM3e memory from Micron. In addition to the new AI platforms, NVIDIA also announced a major supercomputer win with its Grace Hopper Superchips that now power the Exaflop Jupiter supercomputer.

NVIDIA's H100 GPUs are the most highly demanded AI chips in the industry so far but the green team wants to offer even more performance to its customers. Enter, HGX H200, the latest HPC & computing platform for AI which is powered by H200 Tensor Core GPUs. These GPUs feature the latest Hopper optimizations on both hardware and software & while delivering the world's fastest memory solution to date.

The NVIDIA H200 GPUs are equipped with Micron's HBM3e solution with memory capacities of up to 141 GB and up to 4.8 TB/s of bandwidth which is 2.4x more bandwidth and double the capacity versus the NVIDIA A100. This new memory solution allows NVIDIA to nearly double the AI inference performance versus its H100 GPUs in applications such as Llama 2 (70 Billion parameter LLM). The recent advancements in the TensorRT-LLM suite have also resulted in huge performance gains in a vast number of AI applications.

In terms of solutions, the NVIDIA H200 GPUs will be available in a wide range of HGX H200 servers with 4 and 8-way GPU configurations. An 8-way configuration of H200 GPUs in an HGX system will provide up to 32 PetaFLOPs of FP8 compute performance and 1.1 TB of memory capacities.

The GPUs will also be compatible with the existing HGX H100 systems, making it easier for customers to upgrade their platforms. NVIDIA partners such as ASUS, ASRock Rack, Dell, Eviden, GIGABYTE, Hewlett Packard Enterprise, Ingrasys, Lenovo, QCT, Wiwynn, Supermicro, and Wistron, will offer updated solutions when the H200 GPUs become available in the 2nd quarter of 2024.

In addition to the H200 GPU announcement, NVIDIA has also announced a major supercomputer win powered by its Grace Hopper Superchips (GH200). The Supercomputer is known as Jupiter and will be located at the Forschungszentrum Jülich facility in Germany as a part of the EuroHPC Joint Undertaking and contracted to Eviden and ParTec. The supercomputer will be used for Material Science, Climate Research, Drug Discovery, and More. This is also the second supercomputer that NVIDIA announced in November with the previous one being the Isambard-AI, offering up to 21 Exaflops of AI performance.

In terms of configuration, the Jupiter Supercomputer is based on Eviden’s BullSequana XH3000 which makes use of a fully liquid-cooled architecture. It boasts a total of 24,000 NVIDIA GH200 Grace Hopper Superchips which are interconnected using the company's Quantum-2 Infiniband. Considering that each Grace CPU packs 288 Neoverse cores, we are looking at almost 7 Million ARM cores on the CPU side alone for Jupiter (6,912,000 to be exact).

Performance metrics include 90 Exaflops of AI training & 1 Exaflop of high-performance compute. The supercomputer is expected to be installed in 2024. Overall, these are some major updates by NVIDIA as it continues to lead the charge of the AI world with its powerful hardware and software technologies.