Elon Musk’s xAI Using 100,000 NVIDIA H100 GPUs To Train Grok 3 AI Model, Grok 2 To Launch By August
Elon Musk’s xAI Using 100,000 NVIDIA H100 GPUs To Train Grok 3 AI Model, Grok 2 To Launch By August
Elon Musk has revealed that the next-gen Grok 3 LLM model has been trained on a whopping 100,000 NVIDIA H100 AI GPUs.
Well, it looks like every other AI-focused firm is battling each other to push out advanced versions of their LLM models to cater to the growing need for AI chatbots. A new firm to have joined the race is Elon Musk's AI startup, the xAI, which previously announced the development of the "Grok" set of LLM models, designed specifically for its integration as an AI assistant for premium users on X.
The firm has released newer versions of the original model, and xAI is currently preparing for Grok 2; however, Elon, in his latest tweet, has started to promote the next-gen Grok 3 model, claiming that it's going to be much larger than its predecessors.
Grok 3 end of year after training on 100k H100s should be really something special
— Elon Musk (@elonmusk) July 1, 2024
The disclosure of the hardware used to train Grok 3 is something surprising and exciting at the same time because utilizing 100,000 units of NVIDIA's high-end H100 AI GPUs for model training will produce a result that the markets haven't seen yet.
To back this statement, it is rumored that OpenAI's GPT-4 LLM model was trained on 40,000 of NVIDIA's A100 AI GPUs, which are relatively outdated compared to the H100s and lesser in quantity as well; hence, you can imagine the capabilities Grok 3 AI model will hold within itself.
In a previous tweet, Elon Musk claimed that training LLMs through internet data is hectic and requires a "lot of work," especially when it comes to computing resources. He said that the Grok 2 AI model, which is slated for August, is an improvement in this regard and that the next-generation Grok 3 LLM model will build upon the previous models, ultimately being a massive product by xAI.
Apart from that, Elon Musk previously announced that he plans to acquire NVIDIA's flagship Blackwell B200 AI accelerators for xAI as well, apparently valued at $9 billion, which is a gigantic amount considering the dynamics of the markets. With Grok 3 utilizing 100,000 units of NVIDIA's H100s, the model is supposed to cost around $3 billion in just training, justifying the fact that Elon is maxing out on everything with xAI.
It will be interesting to see how xAI and its ventures contribute to the AI hype, and by the looks of it, Elon's AI startup might be the next big thing.
What's Your Reaction?
