NVIDIA’s Explosive AI Growth Leads To Increased GPU Prices, Limited Supply & Massive Order Backlog

NVIDIA is seeing major growth within the AI segment and while that's great, the huge demand has created new problems such as higher GPU prices & limited chip supply.

During the pandemic days, NVIDIA had to deal with huge demand for gaming GPUs in two key segments, crypto, and client. With nothing else to do, demand for gaming hardware saw a big boost but the additional demand in the crypto segment which was quite active during the same period meant that there were limited GPUs to get hands-on, and those that were available were priced ridiculously high.

Now it's the entire opposite, since 2022, gaming GPUs and hardware has been available in great supply but the entire world is facing an economic crisis that has led to a major decline in the PC segment. Furthermore, rising inflation and manufacturing costs have pushed prices of GPUs by a lot and while NVIDIA, AMD, and Intel have tried their best to entice gamers, all recent GPUs have faced consumer backlash, mainly due to higher prices and not offering the same big performance leaps we saw in the previous generations. Furthermore, the crypto flood and older inventories are still there on retail shelves and are often seen as better value than the newer cards. That has also led to prices of existing launches falling below MSRP within a week of their launch.

So while gaming revenue has taken a major blow, AI has come as a savior for NVIDIA and pushed the company to unimaginable heights. ChatGPT, Generative AI, Stable Diffusion, these are some of the buzzwords that you all must be familiar with and NVIDIA has a key role to play in each of these technologies. NVIDIA GPUs are being used to tackle AI, ML & DNN challenges in various workloads and this is just the beginning.

ByteDance, the creators of TikTok, has alone acquired $1 Billion worth of NVIDIA AI GPUs for their servers This order includes a total of 100,000 A100 and H800 accelerators. The H800 is the China-exclusive variant that was designed in accordance with the US policy to restrict China's access to high-end chips that could give them a military and tactical advantage. Now the latest reports are coming in that 2024 will see an even bigger demand for NVIDIA AI GPUs.

Currently, the pricing of NVIDIA's H100 GPUs exceeds 250,000 RMB or $35,000 US. It is expected to reach $50,000 US per H100 accelerator given the current market trend. MyDrivers, who cites tech sources within the industry, reveals that as demand continues to grow, orders continue to pile up and by 2024, any new order can take at least half a year or even a full year to be fulfilled. NVIDIA is likely to give big customers priority and given the China ban on the H100 and A100 closing in, more and more tech companies will be placing orders at NVIDIA to acquire their gold AI silicon.

And these are all Chinese companies, there are several other tech giants outside of China that are demanding access to NVIDIA's latest AI GPUs as fast as possible. Competitors to ChatGPT are also running NVIDIA hardware and many more are following.

NVIDIA is also reportedly working on several new H100 GPU SKUs with Videocardz spotting a 64 GB and a 94 GB variant. The highest-end H100 chip packs 120 GB VRAM which is -37.5% lower than the upcoming AMD Instinct MI300X.

The H100 has been out in the market for a while now so it should have better yields but you can never underestimate the chipset design that AMD is gunning for and NVIDIA is yet to adopt for its GPUs. With that said NVIDIA's ace in the spade is its robust & industry-leading software suite which remains unmatched by the likes of Intel and AMD. Both vendors have tried their own thing to catch up to NVIDIA's CUDA and AI software suite but NVIDIA continues to be the leader.