It’s easy to think about AI costs the same way we think about electricity:

You used a certain number of units, so multiply them by the unit price to get the bill.

In AI, that unit has become the token.

But the problem is that the token shown on a company’s bill doesn’t represent the full economic cost of what happened inside the data center.

Behind every request are processors, memory, networks, power, and storage. And as AI shifts from simple question-and-answer exchanges to agents working through dozens of steps, this infrastructure is starting to bear a very different kind of load.

This is where an important paradox emerges:

The cost per token may fall while the AI bill rises.

Why do agents use more tokens?

A traditional conversation might consist of a question followed by an answer.

An AI agent, on the other hand, may read files, search, call tools, analyze results, and then revisit earlier information before taking the next step.

Each stage may process part of the context again.

That’s why Goldman Sachs Research expects global token consumption to rise by about 24 times between 2026 and 2030, reaching 120 quadrillion tokens per month, driven in particular by the spread of agents among businesses and consumers.

This surge is already underway: Google said it went from processing 9.7 trillion tokens per month across its products in May 2024 to more than 3.2 quadrillion in May 2026.

So the question isn’t just how much a token costs, but how many tokens we now consume to complete the same task—or more complex ones.

And what does storage have to do with it?

When a model works with a long context, it needs to retain information that helps it continue its reasoning process.

An important part of this is called the KV Cache. It consists of intermediate data that helps the model avoid recalculating everything from scratch while processing the context.

Traditionally, this process has relied heavily on fast GPU memory. But as contexts grow and large numbers of agents run in parallel, memory becomes a limited and expensive resource.

That’s why Nvidia introduced the CMX Context Memory Storage platform, which adds a high-performance storage layer between GPU memory and traditional storage, using flash media to retain and share context at a larger scale. The company says this architecture can increase token throughput and reduce power consumption for some long-context workloads.

In other words, storage is no longer just a place to put files after the work is done. It is becoming part of the inference process itself.

When SSD prices become part of AI economics

And that’s where a new problem emerges.

If enterprise storage becomes a bigger part of the infrastructure AI requires, changes in storage prices affect operating costs.

In 2026, the memory and storage market came under significant pressure. TrendForce said Enterprise SSD contract prices rose by about 80% in the first quarter due to AI-related demand and supply shortages, with price pressures continuing through the year. In the fourth quarter, it forecast NAND Flash prices overall would rise by about 15–20% quarter over quarter.

This shows why you shouldn’t look at GPUs alone when calculating the cost of AI.

But isn’t the technology getting cheaper?

Yes—and that’s an important point.

Goldman Sachs estimates that advances in semiconductors are reducing computing costs per token by around 60–70% a year for some inference workloads.

But a lower unit cost doesn’t guarantee a lower bill.

If the price per token is cut in half while a company’s consumption increases tenfold because agents are more widely used, total spending will rise.

This is the same phenomenon we see in other sectors: when a resource becomes cheaper and easier to use, we may end up consuming much more of it.

From a flat subscription to usage-based costs

And this is where the management challenge begins.

Harvard Business Review noted that some software companies subsidized the cost of AI features during the initial adoption phase, while the market is gradually moving toward models that tie costs more closely to actual usage.

That’s why the financial question for companies in the future won’t be:

How much does a million tokens cost?

It will be:

How often does the agent run? How much context does it reread? How much memory and storage does it need? And what economic value does it generate in return?

The real shift in AI economics may not simply be a rise or fall in the price per token.

It may instead be the transition from a stage when AI was an add-on feature in software to one where it becomes an operational resource whose returns must be measured and whose consumption must be managed like any other company cost.