AI Token Spending Jumps 160% as Inference Prices Fall
Venture firm a16z reports that average AI token prices dropped 67% since May 2026, while total token spending surged significantly.
AI Token Spending Jumps 160% as Inference Prices Fall
Venture firm a16z reports that average AI token prices dropped 67% since May 2026, while total token spending surged significantly.
The Rise of AI Token Spending
Artificial intelligence technology has experienced a significant shift as cheaper inference drives higher total consumption. According to data published by venture firm a16z in its Charts of the Week on October 31, 2026, average AI token prices have fallen 67% since May 2026. Despite this dramatic drop in unit costs, total token spending has climbed between 122% and 160%.
In this context, a token serves as the billing unit for AI model inference, representing a small chunk of text that a model reads or writes. Providers charge users directly by the token. The recent data highlights a striking divergence where per-token prices dropped by two-thirds while total expenditure more than doubled, as usage grew fast enough to offset the price reductions entirely.
The Impact of Autonomous AI Agents
The primary driver behind this surge in consumption appears to be the growing reliance on AI agents. Data from a16z indicates that autonomous agents consume nearly five times as many tokens as individual human users. While a human user typically asks a chatbot a direct question and reads the response, an agent executes longer tasks that carry extensive context and require multiple steps, with every step burning through tokens.
Furthermore, agent token usage has expanded roughly 14 times since February 2026. This trajectory points toward a broader industry shift favoring larger, highly automated workflows over simple, one-off prompts. A16z attributes this phenomenon to the Jevons paradox, where making a resource more efficient and cheaper to use ultimately causes total consumption to rise rather than fall.
Hardware Costs and Market Dynamics
While inference prices continue to fall, the hardware market presents a contrasting trend. Rental prices for Nvidia graphics processing units have moved sharply upward since mid-July 2026. Rental costs for Nvidia’s B300 chips rose 106%, while B200 rental prices climbed 66% over the same period.
This dynamic creates an unusual environment for inference providers, who face collapsing revenue per token alongside rising hardware rental costs. The gap between these diverging curves has been described as a rack spread. Conversely, cloud operators and GPU owners benefit from soaring demand for compute capacity that currently outruns available supply.
About The Author
