$11 billion.

On July 8, AI chip company SambaNova announced the first close of its $1 billion Series F round at an $11 billion valuation. General Atlantic led the round, with an all-star cast including BlackRock, Capital Group, Qatar Investment Authority, and Vista Equity Partners participating, and Intel doubling down — this coming just five months after SambaNova closed a $350 million Series E and launched its SN50 chip in February.

More notable than the funding was the customer: JPMorgan Chase selected SambaNova as an inference infrastructure partner, with SN40L and SN50 systems set to power secure on-premises AI inference at the world's largest bank. SambaNova CEO Rodrigo Liang didn't mince words: "Having JPMorgan Chase decide they're going to use SambaNova for their inference solution is a big deal. It sends a message to the banking industry that it's time not to completely depend on cloud services."

That same day, two pieces of news from China formed a perfect echo. InfoQ reported that DeepSeek had started developing its own AI inference chips about a year ago and was currently engaging foundries and storage vendors. A 36Kr dispatch said Hygon Information would showcase its full-stack "cloud-edge-device" compute portfolio, making a major push into edge AI for the first time.

Train on NVIDIA, inference on the public cloud — this default AI infrastructure playbook that held for three years is being completely rewritten in the summer of 2026.

Why Is JPMorgan Ditching Cloud for On-Prem?

The significance of SambaNova winning JPMorgan cannot be overstated.

Banking is the industry with the most urgent AI needs and the strictest security requirements. JPMorgan manages over $3.9 trillion in assets, processes hundreds of millions of transactions daily, and customer data sensitivity is at the highest possible level. For the past two years, JPMorgan, like other Wall Street giants, primarily accessed AI capabilities through the big three cloud providers — AWS, Azure, and Google Cloud — with OpenAI on Azure and Anthropic deployed across AWS and GCP.

But big banks are collectively shifting to private deployments. Liang identified three categories of SambaNova customers — sovereign clouds, neoclouds, and large enterprises building for their own use — with JPMorgan representing the third and most indicative category. The reasons are threefold:

First, data security and compliance. For banks' most sensitive risk models, anti-money-laundering systems, and customer data, many CIOs and CISOs still can't sleep easy even with VPC-isolated cloud. On-premises deployment means data never leaves the bank's own data centers, satisfying the strictest regulatory requirements. Europe's GDPR, U.S. bank secrecy laws, and China's Data Security Law are all pushing financial institutions to bring core AI capabilities back on-site.

Second, the non-linear growth of inference costs. Training is a one-time investment; inference is a continuous cost center. In 2025, the industry was still focused on how much it cost to train large models; in 2026, everyone's doing the math: when models are truly deployed in production with hundreds of millions of calls per day, inference costs grow linearly — even superlinearly — with usage. While cloud inference pricing keeps falling, for hyperscale users the TCO (total cost of ownership) of building private infrastructure is already significantly lower than public cloud on a 2–3 year horizon.

Third, performance advantages of heterogeneous computing. SambaNova's SN50 chip is purpose-built for Agentic Inference — not for training trillion-parameter models, but for making trillion-parameter models run faster with lower latency and cheaper per-token cost during inference. Liang describes SambaNova's core competitive edge as "premium inference": fitting multi-trillion-parameter models onto a single rack and making them run fast. This is fundamentally different from GPU's general-purpose architecture — GPUs are Swiss Army knives that do everything; SambaNova's RDU (Reconfigurable Dataflow Unit) is a scalpel, optimized specifically for inference.

China's Inference Chip Race: DeepSeek Goes Custom, Hygon Pushes Edge

The same day SambaNova won JPMorgan in North America, two important signals emerged from China's AI inference chip race.

First, DeepSeek was reported to be developing its own inference chips. InfoQ's coverage stated that DeepSeek started the AI inference chip project about a year ago and is currently engaging foundry and storage partners. This news shouldn't surprise anyone. From V3 to R1 to its latest models, DeepSeek has consistently pursued extreme cost-efficiency — open source, cheap, low inference cost. But that "cheapness" sits atop NVIDIA GPUs. When your call volume reaches a certain scale, GPU procurement costs and power consumption become the single largest cost line item. By building custom inference chips, DeepSeek can take full control of inference cost economics.

This follows the same logic as Anthropic partnering with Samsung for custom chips and OpenAI building Jalapeño. When AI companies reach a certain scale, not building chips means ceding the throat of your profit margins to NVIDIA and TSMC. But DeepSeek's path may be more aggressive: Anthropic and OpenAI are working with established foundries like Samsung; DeepSeek, as a Chinese company, faces far more complex foundry options and supply chain constraints.

Second, Hygon Information is pushing into edge AI. 36Kr reported that Hygon will showcase its full-stack "cloud-edge-device" compute portfolio at the 2026 Heguang Organization Intelligent Computing Applications Conference, marking Hygon's first large-scale presentation of edge-side compute deployments for industrial and vertical applications, building on its existing CPU and DCU (Deep Learning Processing Unit) dual-chip foundation. Edge AI's core requirements — real-time response, local processing, security and trustworthiness — are precisely why many industrial and edge computing scenarios are reluctant to use the cloud.

Korean AI chip startup Rebellions — backed by Samsung Electronics — also revealed plans to IPO in Korea in Q1 or Q2 2027, and is already generating real revenue. Rebellions is also pursuing a dedicated inference chip path. From SambaNova and Etched in the U.S., to DeepSeek's custom silicon and Hygon's full-stack in China, to Rebellions in Korea, the global AI inference chip landscape is shifting from a NVIDIA monopoly to a diverse, multi-region, multi-architecture, multi-scenario playing field.

The Inference-Era Compute Landscape: From "One Superpower" to "Heterogeneous Warring States"

We are witnessing a fundamental restructuring of AI compute infrastructure.

The training era (2020–2025) had a simple logic: Scaling Laws dictated that whoever could buy the most H100s/H200s/B100s could train the strongest models. NVIDIA was the undisputed king of this era — CUDA ecosystem moats, TSMC advanced process priority, NVLink high-speed interconnects left competitors with virtually no opening. AMD's MI300 series had competitive performance but a massive ecosystem gap; Google TPUs were for internal use only; most AI chip startups remained at the PowerPoint stage.

But the inference era plays by completely different rules. First, inference doesn't need the most advanced process nodes. Training demands extreme compute density and memory bandwidth; inference has far lower process requirements, and mature nodes can deliver competitive performance-per-watt. This opens the door for more players. Second, inference scenarios are highly fragmented. Cloud inference, private data center inference, edge inference, on-device inference — each scenario has different latency, power, and cost requirements, and no single chip can serve all scenarios well. Third, software stack barriers are falling. Inference frameworks like PyTorch, ONNX Runtime, vLLM, and TensorRT-LLM are maturing rapidly, drastically reducing software adaptation costs for new chips.

SambaNova's SN50 choosing deep collaboration with Intel Xeon on a heterogeneous approach, rather than building a standalone chip from scratch, embodies this trend. Liang repeatedly emphasized the Intel partnership in his interview — leveraging Intel's manufacturing scale, supply chain, and customer channels means SambaNova doesn't need to build an entire system from zero. Intel also needs SambaNova — Xeon CPUs need dedicated accelerator chips for AI inference to compete effectively against NVIDIA Grace Hopper.

Microsoft was also recently reported to be joining the AI cost-cutting trend by relying more on its own models. TechCrunch reported that Microsoft is reducing the proportion of OpenAI model calls it uses, shifting toward its in-house Phi series and MaaS models. This sends an equally clear signal: even OpenAI's largest investor is diversifying for inference costs — nobody wants to be locked into a single vendor.

Endgame: A Three-Layer Cake Is Forming for Inference Infrastructure

Pulling these threads together, the endgame outline for AI inference infrastructure is already clear: a three-layer cake.

Layer one: hyperscale public cloud inference. AWS, Azure, GCP, Alibaba Cloud, Tencent Cloud will continue providing general-purpose inference serving SMBs, elastic workloads, and non-sensitive scenarios. The core competitive advantages here are scale and ease of use. NVIDIA GPUs will remain dominant in this layer but will see share fall from near-monopoly to 50–60%.

Layer two: industry/sovereign private inference infrastructure. Banks, governments, healthcare, and large manufacturers will increasingly build their own private AI inference clusters. Dedicated inference chip companies like SambaNova, Etched, and Rebellions, along with domestic compute vendors like Hygon and Huawei Ascend, will compete head-to-head with NVIDIA in this layer. Core competitive advantages here are security and control, TCO, and scenario-specific performance optimization. JPMorgan choosing SambaNova is the starting gun for this layer's explosion.

Layer three: on-device and edge inference. Phones, PCs, cars, industrial equipment, robots — these scenarios require ultra-low latency, extreme privacy, and offline availability. Models will increasingly run on-device rather than in the cloud. Qualcomm, MediaTek, Apple, Hygon, and Horizon Robotics will compete fiercely in this layer.

For AI industry practitioners and investors, this structural shift sends several clear signals. First, inference chips are a bigger market than training chips — training tops out at tens of billions annually; inference is a hundreds-of-billions, even trillion-dollar market that lasts far longer. Second, "full self-development" isn't the only path — SambaNova's Intel partnership proves heterogeneous computing and strong alliances may be more efficient than vertical integration. Third, the window is opening for Chinese inference chip vendors — DeepSeek's custom silicon and Hygon's full-stack push aren't about beating NVIDIA at training; they're about capturing domestic market share in the much larger inference market.

The $11 billion valuation is just the beginning. The independence war for AI inference infrastructure has just fired its first shot.

See you tomorrow.