InvestAI etc. - 6/25/2024
AI for investors, explained in simple terms. An open thread updated weekly.
Topics discussed this week:
AI-generated videos and next generation GPUs
The proprietary software monopoly at the heart of Nvidia’s success
AeroVironment
LLM-composed op-eds
AI Cheating
AI-generated videos and next generation GPUs
>
SAMI >
So far we have spoken mainly about LLMs (large language models such as ChatGPT, Gemini etc.) and their various text or graphic functions but there are numerous other applications in the works that are going to require a much larger number of GPUs (graphic processing units). Video is the next obvious one, given the popularity of YouTube, Instagram, TikTok etc. The younger generations don’t read as much as the older and they prefer to get information through video.
>
RICHARD >
Yes, there are estimates that a single H100 (Nvidia's high-end GPU) can generate about one hour of Sora quality video per day. If AI-generated video gains significant traction among the YouTube/Tiktok set, that would result in a big increase in H100 revenues. Check out this blog post about Sora > > > Under The Hood: How OpenAI's Sora Model Works
>
SAMI >
It’s a good overview. Here is an excerpt that gives a summary:
Sora is a diffusion model… Broadly speaking, diffusion models are a type of generative machine learning model that learns to create data resembling the data they were trained on, such as images or video, by gradually learning to reverse a process that adds random noise to data. Initially, these models start with a pattern of pure noise and step-by-step remove this noise, refining the pattern until it transforms into coherent and detailed output.
>
RICHARD >
This photo from the article sums up the diffusion process:
>
SAMI >
On a related note, Nvidia sold an estimated 1.5 million units of the H100 in 2023 of which 300,000 were to Meta and Microsoft. And I am guessing not a lot of these were for video applications.
>
RICHARD >
At $30K/each, that's a lot of revenue. And that's the old generation. The new B200 GPUs shipping later this year were designed after the big explosion of interest in GPT. So, Nvidia put in lots of optimizations that will make B200s must-have chips for the big AI companies.
>
SAMI >
The Verge outlined in March the advantages of B200s over H100s in terms of performance, time, cost, and energy usage. Here is an excerpt:
Nvidia says the new B200 GPU offers up to 20 petaflops of FP4 horsepower from its 208 billion transistors. Also, it says, a GB200 that combines two of those GPUs with a single Grace CPU can offer 30 times the performance for LLM inference workloads while also potentially being substantially more efficient. It “reduces cost and energy consumption by up to 25x” over an H100, says Nvidia, though there’s a question mark around cost — Nvidia’s CEO has suggested each GPU might cost between $30,000 and $40,000.
Nvidia can charge whatever they want, given their quasi-monopolistic position. As they used to say about IBM, no one is going to get fired for buying from Nvidia instead of another supplier. But it’s important to realize that there are in fact other suppliers out there, with competitive products at lower costs.