From Training to Inference – Why AI Inference Is Reshaping the Global Chip Market and What It Means for China

For the past several years, the AI industry has been defined by one word: training. Building ever-larger models consumed the vast majority of computing power, and NVIDIA rode that wave to a trillion-dollar market cap. But 2026 marks a fundamental shift. The AI industry is moving from training models to deploying them — and that changes everything. This is the year inference takes over.

1. The Structural Shift: From Training to Inference

To understand why this matters, it helps to distinguish between the two phases of AI computing. Training is the process of building a model — feeding it vast amounts of data over weeks or months until it learns patterns. It is a one‑time, capital‑intensive investment. Inference, by contrast, is what happens when that model is deployed to answer questions, generate content, or take actions in real time. It is ongoing, operational spending — and it scales with usage.

For years, training dominated. Lenovo CEO Yuanqing Yang noted that until recently, approximately 80% of AI compute went to training and only 20% to inference. That is now reversing. Yang forecasts that within the next few years, 80% will be inference and 20% training. Deloitte estimates that inference workloads accounted for half of all AI compute in 2025 and will jump to two‑thirds in 2026.

2. Why Inference Is Exploding: AI Agents and the Token Tsunami

The primary driver of this shift is the rise of AI agents — autonomous systems that can plan, execute tasks, and use tools. Unlike traditional chatbots that handle a single query in a single turn, agents perform multi‑step reasoning, tool calls, and long‑context memory. The result: a single agent task consumes 4 to 15 times more tokens than a standard conversation. Gartner estimates that an agentic workflow uses 5 to 30 times more tokens per task than a standard chatbot.

The numbers are staggering. China’s total daily token consumption grew from 100 billion in early 2024 to 180 trillion by February 2026 — an 1,800‑fold increase in just over two years. Globally, Goldman Sachs projects that token consumption will multiply by 24 times between 2026 and 2030, reaching 120 quadrillion tokens per month.

This explosion in token consumption is fundamentally an inference problem. Agents are deployed — they are not trained in real time. Every task they perform is an inference workload. As one industry observer put it, 2026 is the “year of the AI agent”, and that means inference demand is entering a period of exponential growth.

3. What Inference Demands — and Why It Changes the Chip Game

Inference workloads are fundamentally different from training workloads — and that difference creates new opportunities for chipmakers.

⚙️ Less Demanding on Process Technology

Training requires massive parallelism and peak floating‑point performance, which pushes chips to the leading edge of semiconductor manufacturing. Inference, by contrast, is more about throughput, latency, and power efficiency than raw peak compute. This means inference chips can often be built on less advanced process nodes — a critical advantage for Chinese manufacturers that face restrictions on accessing the most advanced EUV lithography.

🔓 Weaker Ecosystem Lock‑in

Training has been dominated by NVIDIA’s CUDA ecosystem, which has locked in developers for nearly two decades. Inference, however, is a more fragmented landscape. Many inference workloads run on CPUs, custom ASICs, or specialised accelerators — and the software stack is less monolithic. This creates a window of opportunity for alternative architectures to gain a foothold.

🎯 Specialisation Matters

Inference workloads vary widely. A gaming application may require a 15‑millisecond first‑token latency, while a customer service bot can tolerate 100 milliseconds. This diversity means that one size does not fit all — and that creates space for customised, application‑specific chips that can outperform general‑purpose GPUs in specific inference scenarios.

4. The Inference Opportunity for Chinese Chipmakers

The shift to inference is arguably the single biggest opportunity for Chinese chipmakers in the AI era. Here is why.

📈 Domestic Inference Share Is Rising Fast

China’s inference chip market is estimated to reach approximately $11.86 billion (¥86 billion) in 2026, representing 28.7% of the global market. Domestic inference chips now account for about 34.6% of shipments in China, up from 24.8% in 2025. Some estimates put the overall domestic inference chip market share at 45‑55%, with NVIDIA’s share in China’s cloud inference segment falling to just 8‑20%.

🏆 The Domestic Contenders

These four players — Huawei Ascend, Cambricon, Alibaba T‑Head, and Baidu Kunlunxin — form the core of China’s domestic inference chip ecosystem. In 2025, domestic AI accelerators accounted for 41% of the 4 million AI GPUs delivered to China’s AI server market. That share is rising rapidly in 2026.

🧩 ASIC: The Inference‑Optimised Architecture

The rise of inference is closely tied to the rise of ASICs (Application‑Specific Integrated Circuits). Unlike general‑purpose GPUs, ASICs are designed for specific workloads — and inference is a perfect fit. TrendForce projects that ASIC AI server shipments will grow from 27.8% of total AI servers in 2026 to nearly 40% by 2030. Goldman Sachs expects ASICs to account for 40% of AI chips by 2026. Some analysts project ASIC shipments of approximately 7.7 million units in 2026, representing 45% of the AI accelerator market, and forecast that ASICs will surpass GPUs in share by 2027.

5. What This Means for Overseas Businesses

If your company has supply chain relationships, investments, or partnerships in China’s tech sector, the inference shift has direct implications for you.

• Chinese partners are transitioning to domestic inference chips. The combination of US export controls and domestic procurement mandates means that many Chinese firms are actively replacing NVIDIA GPUs with domestic alternatives for inference workloads. This affects not just chip procurement, but also software stacks, deployment timelines, and operational costs.
• New ecosystem partners are emerging. The rise of domestic inference chipmakers — Huawei, Cambricon, Alibaba, Baidu — means that new suppliers, software vendors, and system integrators are gaining prominence. Overseas firms need to map this new landscape.
• Compliance requirements are evolving. With Chinese policies mandating domestic chip procurement in data centres and state‑funded projects, the compliance landscape is shifting rapidly. What was acceptable six months ago may not be today.
• Due diligence must be updated. A partner’s financial health, regulatory compliance, and supply chain resilience are all affected by the chip transition. Outdated due diligence is a significant blind spot.

6. How ChinaBizInsight Can Help You Navigate the Shift

At ChinaBizInsight, we specialise in providing overseas businesses with reliable, verifiable information about Chinese companies. As the AI chip market undergoes its most dramatic transformation in decades, the need for accurate due diligence has never been greater.

We help overseas businesses:

• Verify Chinese company credentials — including business licences, shareholder structures, and director information — through official government sources.
• Access comprehensive credit reports that go beyond basic registration to include legal risks, financial health, and operational history.
• Obtain notarisation and apostille services for Chinese corporate documents, ensuring they are recognised in your home jurisdiction.
• Conduct specialised due diligence on companies in high‑tech sectors, including AI chip design, semiconductor manufacturing, and related supply chains.

Final Take — The Inference Era Has Arrived

The shift from training to inference is not a future trend — it is happening now. In 2026, inference compute is growing at more than double the rate of training. Inference workloads will account for two‑thirds of all AI compute. AI agents are driving exponential growth in token consumption. And Chinese chipmakers are capturing an expanding share of the inference market.

For overseas businesses, this transformation brings both risks and opportunities. The key to navigating this new landscape is reliable, up‑to‑date information about your Chinese partners and counterparties. Know who you are doing business with — because the landscape has changed, and it will never be the same.