TensorWave Believes It Holds the Key to Breaking Nvidia's Dominance in AI Compute with an AMD-Powered Cloud

Chipmaker Nvidia achieved $30 billion in revenue in the last fiscal quarter, driven primarily by the high demand for GPUs in the AI industry. GPUs are crucial for training and running AI models, as they contain thousands of cores that work simultaneously to swiftly perform the necessary linear algebra equations supporting the models. The hunger for AI technology remains strong, with Nvidia’s GPUs becoming the preferred choice among AI players of all sizes. However, TensorWave, a company established recently, is taking a different approach by launching a cloud service that exclusively provides access to hardware from Nvidia’s competitor AMD for AI workloads.

Winding Paths

The inception of TensorWave stemmed from a casual gathering after a pickleball match that brought together the company’s co-founders, Jeff Tatarchuk, Piotr Tomasik, and Darrick Horton. Discussion around the monopolistic control over GPU compute capacity and resulting supply constraints led to the formation of TensorWave. The founders, who shared a background in technology and venture capital, believed that they could address the GPU supply issue and provide a competitive alternative in the AI market.

Vegas, Inc.

Despite being an unconventional choice for a cloud infrastructure startup, TensorWave is headquartered in Las Vegas due to the team’s optimism about the city’s potential as a thriving technology and startup hub. Las Vegas is home to over 600 startups employing more than 11,000 individuals, attracting substantial investments in recent years. With lower energy costs and overhead compared to major U.S. cities, Las Vegas offers a conducive environment for tech ventures. The founders’ connections to the city’s venture capital community facilitated TensorWave’s establishment as one of the early cloud providers offering AMD Instinct MI300X instances for AI workloads. The company differentiates itself by providing dedicated storage, high-speed interconnects, and competitive pricing in the cloud space, focusing on AI-specific compute solutions.
MI300X: racked, stacked, and ready for production! Let’s ROCm roll! #TensorWave #AMD #GPUrich #First pic.twitter.com/QAyHMJawNr

First, on price. Horton notes that the MI300X is significantly cheaper than Nvidia’s most popular GPU for AI workloads at present, the H100, and that this allows TensorWave to pass savings on to customers. He wouldn’t reveal TensorWave’s exact instance pricing. But to beat the more competitive H100 plans, it would have to come under ~$2.50 per hour — a challenging but not inconceivable feat.

“Pricing ranges from approximately $1 per hour to $10 per hour, depending on the bespoke requirements of the workload and the GPU configurations chosen,” Horton said. “As for the cost per instance that TensorWave incurs, we are unable to share those details due to confidentiality agreements.”

Second, on performance. Horton points to benchmarks showing the MI300X outgunning the H100 when it comes to running (but not training) AI models, specifically text-generating models like Meta’s Llama 2. (Other evaluations suggest that the advantage may be workload-dependent.)

There seems to be some credence to Horton’s claims, given interest from tech industry movers and shakers in the MI300X. Meta said in December that it’ll use MI300X chips for use cases like running its Meta AI assistant, while OpenAI, the maker of ChatGPT, plans to support the MI300X in its developer tooling.

The competition

Others placing bets on AMD’s AI chips range from startups like Lamini and Nscale to larger, more entrenched cloud providers such as Azure and Oracle. (Google Cloud and AWS remain unconvinced of AMD’s competitiveness.)

Working in all of these vendors’ favor right now is the continued Nvidia GPU shortage and the delay of Nvidia’s upcoming Blackwell chip. But the shortage could ease soon with a ramp-up in the manufacturing of critical chip components, in particular memory. And that could allow Nvidia to scale up shipments of the H200, the H100’s successor, which boasts dramatically improved performance.

Another existential dilemma for upstart clouds betting on AMD hardware is bridging the competitive moats Nvidia has built around AI chips. Nvidia’s development software is perceived as more mature and easier to use than AMD’s — and it’s widely deployed. AMD CEO Lisa Su has admitted that it “takes work” to adopt AMD.

On the far horizon, competing on pricing might become challenging down the line as hyperscalers increase their investments in custom hardware to run and train models. Google offers its TPUs; Microsoft recently unveiled two custom chips, Azure Maia and Azure Cobalt; and AWS has Trainium, Inferentia and Graviton.

“As developers seek alternatives that can effectively handle their AI workloads, especially with increased memory and performance demands, along with ongoing production issues causing delays, AMD will maintain superiority for even longer, playing a key role in the democratization of compute in the AI era,” Horton said.

Early demand

TensorWave began onboarding customers late this spring in preview. But it’s already generating $3 million in annual recurring revenue, Horton says. He expects that figure will reach $25 million by the end of the year — an 8x leap — once TensorWave ratchets up capacity to 20,000 MI300Xs.

Assuming $15,000 per GPU, 20,000 MI300Xs would amount to a $300 million investment — yet Horton claims TensorWave’s burn rate is “well within sustainable levels.” TensorWave previously told The Register that it would use its GPUs as collateral for a large round of debt financing, an approach employed by other data center operators, including CoreWeave; Horton says that’s still the plan.

“This reflects our strong financial health,” he continued. “We’re strategically positioned to weather potential headwinds by delivering value where it’s most needed.”

I asked Horton how many customers TensorWave has today. He declined to answer due to “confidentiality,” but highlighted TensorWave’s publicly announced partnerships with networking backbone provider Edgecore Networks and MK1, an AI inferencing startup founded by ex-Neuralink engineers.

“We are rapidly expanding our capacity, with multiple nodes available, and we are continually increasing capacity to meet the growing demands of our pipeline,” Horton said, adding that TensorWave plans to bring AMD’s next-gen MI325X GPUs, which are scheduled to be released in Q4 2024, online as early as November/December.

Investors seem pleased with TensorWave’s growth trajectory so far. Nexus VP revealed on Wednesday that it led a $43 million round in the company, which also had participation from Maverick Capital, StartupNV, Translink Capital, and AMD Ventures.

The tranche — TensorWave’s first — values the startup at $100 million post-money.

“AMD Ventures shares TensorWave’s vision to transform AI compute infrastructure,” AMD Ventures SVP Mathew Hein said in a statement. “Their deployment of the AMD Instinct MI300X and ability to offer public instances to AI customers and developers positions them as an early competitor in the AI space, and we are excited to support their growth through this latest round of funding.”