When a company develops a new product, it usually tries to bring it to market before its competitors. Early access can mean more customers, a larger market share, and a higher return on investment.

But what happens when the new technology itself is capable of creating risks that extend beyond the company and its users?

This question has become more important as AI models evolve and gain the ability to use computers and carry out multiple tasks with greater autonomy.

In September, OpenAI Chief Scientist Jakub Pachocki wrote that the world is moving toward more intelligent machines, and that maintaining human control over this transformation requires shared safety standards and international coordination—and, in some cases, even a voluntary slowdown until better safeguards are available.

But the issue here is not only technical; it is economic as well.

When a Company Does Not Bear the Full Cost of Risk

In economics, there is a concept known as externalities.

It occurs when one party gains a benefit or incurs a cost, but part of that cost falls on other people.

If a company develops a more capable model and profits from selling it, while its misuse—for example, leads to higher cyberattack costs for other companies and institutions, the developer does not necessarily bear the full cost created by the product.

This is where the problem arises.

The market rewards a company for speed, capability, and reach, while other parties may bear part of the risks.

As models become more capable, the economic question becomes increasingly important: how can we align the commercial incentive to develop the technology with the social incentive to make it safe?

Why Is It Not Enough for One Company to Slow Down?

Suppose two companies are competing to develop the most advanced model.

If the first decides to slow development to conduct additional safety testing, while the second continues at full speed, the first may lose part of its market share.

The companies may therefore find themselves facing what is known as a collective-action problem.

Each company may recognize that stricter safeguards would benefit everyone, but fear bearing the cost of compliance alone while competitors continue to advance.

That is why Pachocki called for shared minimum safety standards and international coordination, rather than relying solely on individual decisions made by each company.

Why Has the Issue Become More Urgent?

The reason is that the capabilities of the models themselves are changing.

OpenAI recently released GPT-6 Astra, which the company classified as its first model to reach the Critical level in cybersecurity capabilities under its preparedness framework. According to the company’s assessment, this means that under certain conditions, the model can discover unknown security vulnerabilities and develop methods to exploit them without requiring human guidance at every step.

These capabilities could be highly useful to cyber defenders, but they also illustrate why the cost of error rises as the technology advances.

The more capable the tool, the greater the economic value it can potentially create—but the scale of harm may also increase if it is misused.

Where Does Regulation Come In?

Here, regulation works in a way similar to the rules found in sectors such as banking, aviation, and pharmaceuticals.

The goal is not to prevent innovation, but to establish a minimum level of risk that society can accept.

The European Union offers an early example of this. General-purpose models classified as posing systemic risks are subject to requirements that include assessing and mitigating risks, reporting incidents, testing models, and securing them against cyber risks. Full enforcement powers for these obligations began in August 2026.

From a Race for Speed to a Race for Trust

AI offers enormous economic potential—from increasing productivity to accelerating scientific research and reducing the cost of carrying out tasks that previously required entire teams of specialists.

But as systems become more capable of operating autonomously, the question is no longer only: Who has the most powerful model?

It has also become: Who can prove that the most powerful model can be operated reliably and safely?

This is where safety itself may become part of economic competition.

In the early stages of any technology, speed is an advantage. But when the technology becomes essential to companies, governments, and infrastructure, trust becomes an advantage no less valuable than capability.

That is why the next challenge for AI may not simply be to build a more intelligent model, but to build the rules and institutions that allow us to use this intelligence without its capabilities advancing faster than our ability to manage it.