The Explosive Growth of AI Accelerators
The Semiconductor Startup Renaissance Part 2
The Semiconductor Startup Renaissance: Excitement and investment in the semiconductor space has dramatically increased in the past few years. Drawing on our deep knowledge of the sector and extensive portfolio of successful semiconductor investments at Celesta Capital, this article series explores the major themes driving renewed interest in chip innovation.
For more context, see Part 1: “What’s Next After Moore’s Law?”
Long before Nvidia was a titan of U.S. markets, it was a startup supplier of graphics cards (GPUs). GPUs hold an edge over traditional processors like Intel CPUs because graphics processing is “embarrassingly parallel” – you can divide the problem into arbitrary subsections and compute them at the same time versus traditional serial processing, which is slower. Nvidia has compounded this edge by building out the CUDA software platform which allows programmers to access this parallel processing power in a variety of languages without needing to understand the underlying Nvidia hardware/firmware.
As it turns out, there are many other types of workloads that benefit from parallel processing. Thanks to Nvidia’s excellent hardware and investment in CUDA, their GPUs became the default processor for several explosive markets as they emerged.
If you are a technology company leader, there are many good reasons to choose Nvidia in the formation and growth stages of a market:
Nvidia improves silicon performance on a product cadence that is predictable and well-communicated.
Nvidia is a stable company that is more likely than a startup to support your product well into the future.
Nvidia’s CUDA is a robust software platform that continues to expand its breadth and depth.
Their CEO wears leather jackets and gives inspiring speeches about AI factories and robotic futures.
But is Nvidia really too big or too far ahead to disrupt at this point, as many portend?
One key issue to keep in mind is that for these emergent markets, Nvidia GPUs are general purpose processors. All else being equal, a general-purpose processor will always be less performant for a specific task than a specialized processor. As Moore’s law slows and the AI market grows, ample opportunity is opening up for the development of specialized processors.
A good way to test the potential opportunity of building a specialized processor for a market is to gauge the overall market size and the dynamism of the problem space.
It’s easier to build a specialized processor to solve a single type of problem that never changes; it’s much harder to build one for a problem that is constantly changing. Designing and fabricating a new processor is an incredibly expensive undertaking, so the resulting market size and profit pool must justify that investment.
A simple example where the investment was worth it is Bitcoin mining. The computation required to mine Bitcoin is well-understood, never changes, and highly parallel. Nvidia and other GPUs were a common choice for Bitcoin mining in its early days. Over time the market size has grown immensely and the static nature of the problem space attracted competition. Now, companies like Auradine have developed specialized processors with far better performance than even the most expensive GPUs.
This brings us to AI. There is no question that AI is a dynamic space. It seems that what constitutes “state-of-the-art” changes daily as new research and models are released. But the AI market was large enough to attract specialized competition, even long before ChatGPT. SambaNova and Habana, along with others in the first wave of AI silicon startups, created specialized processors addressing the entire AI market. Today, Sambanova’s Samba-1-Turbo set a new record for large language model inference performance in recent benchmarking by Artificial Analysis, and Habana’s Gaudi 3 AI processor is expected to generate more than $500M of revenue for its acquirer, Intel.
Even accounting for these early pioneers, the real explosion in the number of AI accelerators has come post-ChatGPT. As the market grapples with a potentially much larger overall AI market, individual submarkets are considered large enough to justify specialized processor companies. Celesta PortCo Recogni is targeting the Generative AI Inference market. Other companies take this to the logical extreme. For example, Taalas aims to turn individual AI models into custom silicon.
Looking ahead, we can expect further fragmentation and specialization in AI hardware as the market matures. When SambaNova was founded in 2017, Google had just published its famous Transformer paper, and few predicted its world-tilting impact.
We can only speculate on the future breakthroughs that will transform our understanding of current AI markets. What we do know is that AI is driving entrepreneurs and VCs to invest in silicon for new markets at a rate we haven’t seen in decades.