Nvidia H200
AI GPU with 141 GB of HBM3e memory
The Nvidia H200 is a graphics processing unit designed for artificial intelligence and high-performance computing. Built on the Hopper architecture, it features 141 gigabytes of HBM3e memory. It was announced in late 2023 to accelerate large language models and scientific simulations.
- First GPU to feature 141 GB of HBM3e memory
- Delivers 4.8 TB/s of memory bandwidth
- Based on the Nvidia Hopper architecture
- Accelerates LLM inference up to 2x over H100
- Available in HGX and air-cooled NVL variants
What it is
The Nvidia H200 is a specialised graphics processor intended for data centres. It targets generative artificial intelligence and high-performance computing tasks. The chip uses the Hopper microarchitecture. Its defining feature is the inclusion of 141 gigabytes of HBM3e memory. This memory type offers higher capacity and speed than previous generations. Nvidia states that this design improves energy efficiency and lowers the total cost of ownership for large-scale deployments. The device aims to remove memory bottlenecks that often slow down complex calculations.
How it came about
Nvidia announced the H200 at the SC23 supercomputing conference in November 2023. The company had previously integrated this specific memory configuration into the Grace Hopper GH200 module. That earlier product combined Arm processor cores with the H200 graphics unit. The standalone H200 launch followed to make the technology available for general server use. Production units were expected to reach customers in 2024. This release continued a pattern of memory upgrades seen in prior Nvidia generations, such as the A100 and V100.
Inside the chip
The H200 relies on the Hopper architecture. It connects six stacks of HBM3e memory directly to the processor die. Each stack likely holds 24 gigabytes, totaling 141 gigabytes. This arrangement provides a bandwidth of 4.8 terabytes per second. Nvidia claims this is nearly double the capacity of the H100 model. The increased bandwidth allows faster data transfer for simulations and AI inference. The chip supports NVLink connections to combine multiple units. An NVL variant exists for air-cooled enterprise racks, offering flexible configurations with up to four GPUs linked together.
How it was made
The H200 uses the SXM5 module format. It is typically sold in assemblies containing eight graphics processors. These modules fit into HGX or MGX certified systems. The move to HBM3e memory required changes to the physical packaging to accommodate the larger stacks. The physical layout supports both liquid-cooled and air-cooled server environments, depending on the specific system integration.
Where it went
This processor handles workloads that require massive data movement. It accelerates large language models like Llama2 and GPT-3 during inference. Scientific applications such as GROMACS, CP2K, and Quantum Espresso also benefit from the high bandwidth. Nvidia states that the H200 can deliver results up to 110 times faster than central processing units for certain high-performance computing tasks. Enterprise servers use the NVL version for retrieval-augmented generation and computer vision tasks. The hardware supports production-ready AI solutions through included software subscriptions.
Versions and family
The primary variant is the HGX H200, which uses SXM5 modules. The H200 NVL is designed for lower-power, air-cooled racks. It connects up to four GPUs via NVLink bridges. The Grace Hopper GH200 is a related system-on-chip that pairs the H200 graphics unit with 72 Arm Neoverse V2 cores. This combination is distinct from the Grace Superchip. The H200 serves as the successor to the H100, offering a significant increase in memory capacity without changing the power profile.
Why it matters
The introduction of the H200 pushed memory capacity to new levels for AI training and inference. By doubling the memory available compared to its predecessor, it allowed larger models to run on fewer chips. This shift influences server design, encouraging the adoption of liquid cooling solutions to manage heat in dense configurations. The availability of 141 gigabytes of fast memory helps reduce the total cost of ownership for companies running large-scale AI factories. It sets a new baseline for memory-intensive scientific research.
Questions people ask about Nvidia H200
How much memory does the Nvidia H200 have?
The H200 contains 141 gigabytes of HBM3e memory. This is nearly double the capacity of the previous H100 model. The memory operates at a bandwidth of 4.8 terabytes per second.
What is the difference between the H200 and the GH200?
The H200 is a standalone graphics processing unit. The GH200 is a module that combines the H200 GPU with 72 Arm Neoverse V2 CPU cores. They are not compatible with each other as separate components.
Is the H200 suitable for air-cooled servers?
Yes, the H200 NVL variant is designed for air-cooled enterprise rack designs. It supports flexible configurations with up to four GPUs connected via NVLink bridges for lower-power environments.
When was the Nvidia H200 released?
Nvidia announced the H200 at the SC23 conference in November 2023. The company stated that the chips would become available to customers in 2024.
Good reading and sources
There is no English Wikipedia article on Nvidia H200 yet, so the facts on this page come from the sources listed above: nvidia.com/en-us/data-center/h200/, servethehome.com/nvidia-h200-launched-with-141gb-of-hbm3e-at-sc23/. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryChip wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.