In addition to the NVIDIA ACE artificial intelligence (AI) system, the NVIDIA DGX GH200 supercomputer was also announced by Jensen Huang just now. It is the first supercomputer system in the world with a GPU memory of 144 Terabytes. This also makes it the world's first computer with memory exceeding 100 TB.
According to NVIDIA DGX GH200 contains 256 NVIDA Grace Hopper Superchip systems announced last year with each one equipped with a GPU with 480 GB of LPDDR5 memory. Connection using NVLink-C2C allows memory to be shared between Grace Hopper Superchips quickly. According to NVIDIA it has 7x more bandwidth than PCIe Gen5.
In order for the NVIDIA DGX GH200 to operate, it requires 241 kilometers of optical fiber, equipped with 2112 60mm cooling fans that blow at a rate of 70,000 cubic feet of air per minute and the system has a total weight of 18,000 kilograms.
The performance of the NVIDIA DGX GH200 is more than 2 times that of the DGX H100 system used by their customers such as Micorosoft to train the GPT3 language model (LLM). With faster performance, the process of training next generation AI will accelerate even faster.