Chinese AI Startups Are Opting For GeForce RTX 4090 GPUs As NVIDIA’s H20 Accelerators Offer Poor Value

Due to the ban on high-end data center NVIDIA GPUs, Chinese startups have begun using GeForce RTX 4090 as NVIDIA's alternative isn't cost-effective.

The AI demand keeps increasing but as the Chinese market suffers from the ban on the supply of NVIDIA's top AI chips, many are forced for alternatives. It is reported that many startups in China are opting for NVIDIA's flagship gaming GPU the GeForce RTX 4090 instead of H20.

Since US sanctions on China have banned NVIDIA from selling the top AI GPUs like A100 and H100 in the country, several companies looked for alternatives. Even though NVIDIA did offer the H20 AI GPU, which started its mass production in Q2 2024, the GPU did not attract the Chinese market as thought.

The H20 is a cut-down H100 variant offering 296 SXM TF16 FLOPS of theoretical performance compared to the 1979 SXM TF16 FLOPS of the H100. However, not only is the H20 significantly nerfed in performance, but it does not provide any advantages in the power-efficiency department. Unlike the RTX 4090, the H20 is far more expensive for the performance it brings to the table.

This is why a lot of small companies have started utilizing the RTX 4090, which also has great availability despite the gaming graphics card being banned in the region. It is mentioned that despite the ban, startups are easily able to get several GeForce RTX 4090 GPUs for their needs. Even NVIDIA's GeForce RTX 4090D, a China-exclusive variant, has better value than the H20 AI Accelerators. These GPUs are also being equipped with double the memory to sustain large language models.

Currently, the NVIDIA H20 Accelerators sit at around 1.3M RMB with its main demand in well-established companies like Alibaba, Tencent, Baidu, and Byte. As per another company, the RTX 4090 is the first choice of their customers and they will be aiming to install 100 more RTX 4090 GPUs this month. Another entrepreneur said that his company was previously renting the A100 and H100 through cloud services but the local machines still utilize the V100 and RTX 3090 graphics cards.

It has been around one and a half years since the US govt banned the sales of NVIDIA's top-tier AI chips such as the A100/A800 and the Hopper H100/H800 GPUs in China. Some of these GPUs still find their way through the black market but due to poor availability and high pricing, not many companies, especially startups can get their hands on these GPUs for their systems.

It is not the first time that Chinese AI companies have started using RTX 4090 in place of AI GPUs but since NVIDIA H20 did not enter the market till earlier this year, there was no other alternative. Until we see newer and better AI GPUs from either NVIDIA or its rivals, most startups will stick to the gaming GPU instead. With that said, NVIDIA is still expected to make billions of dollars in total revenue from the H20 accelerators since they are sold in several thousand unit quantities to the larger data center customers whereas the smaller AI startups hardly go above a few hundred units.