Drones are cheap, disposable, and the future of war. Over the past four years, we have seen platforms, missiles, and heavy infantry become increasingly obsolete in the face of $500 drones carrying a pack of explosives—a cost advantage that has let Iranians and Ukrainians alike neuter the conventional capabilities of their great power rivals. Eighty percent of casualties in the bloodiest war since 1945 are from drone strikes, Russia has managed to lose one-third of its fleet to a country without a navy, and the US is spending millions of dollars to intercept five-figure Shaheds flying over the Strait of Hormuz.

All this is the result of a technology that is still immature. The violence inflicted by today’s drones is the handiwork of the scant few that manage to evade countermeasures (a mix of radio jamming, high-power microwave weapons, missiles, automatic cannons, interceptor drones, and nets) before making contact. These defenses exploit the inherent limitations of drones—human guidance, GPS feedback, flight exposure, radio links, range—to take them down en masse. And yet, even though 75% of some types of drones manufactured today never reach their targets, they have nonetheless been strategically decisive in Ukraine and elsewhere.

These limitations will not hold for long. Just as bacteria overexposed to antibiotics evolve resistance, overexposure to counterdrone tech has bred ever-more-autonomous drones. In the process of facilitating this arms race, states are likely to incrementally create and deploy an entirely new class of WMD—one that could provide rogue states with the nonnuclear means to threaten superpowers, or hand terrorists the means to selectively assassinate their political targets or civilians en masse.

Unfortunately, drone weapons intended for mass destruction have few barriers remaining to mass deployment. Even well before they reach the level of autonomy needed to surgically take out hardened targets on the battlefield, drones will be capable of employing their existing ability to navigate interiors, find and track human targets, and deploy simple antipersonnel devices to indiscriminately threaten civilians. Below, we discuss the looming arrival of miniature autonomous weapons, the limits of counterdrone technology, and the applications of drones as weapons of mass destruction.

Breaking the Last Barriers to Autonomous Weapons

The ideal drone weapon is a slaughterbot: a small, fully autonomous weapon system that can independently select and hunt its targets. For the most part, the necessary technology for such weapons already exists: airframes the size of a fist and the capability to track human targets are already on the front lines in the form of reconnaissance drones and semiautonomous weapons like the Russian V2U. Even now, these micro drones are agile and autonomous enough to hunt down and kill small moving targets like mosquitoes—to say nothing of the drone technology advances expected in coming years.

From here, the only barrier to weaponization is integration: improving navigation enough to make drone technology useful for mass homicide in an urban setting, as well as packing the necessary guidance, sensor, and payload technology onto a small and energy-efficient chassis. Regrettably, this seems like less of an engineering problem than one of mission design: so long as the attacker is willing to accept indiscriminate targeting and use simple payloads aimed at civilians, the underlying technology is already—or very nearly—ready for practical use.

To understand how close we are to these kinds of weapons, it helps to understand why we do not already employ fully autonomous drones. Right now, the most deadly drones are small first-person-view (FPV) units, with the majority of Russian and Ukrainian casualties alike stemming from direct drone strikes. For the most part, however, these small drones are being piloted directly by humans, either through a radio link or a spool of fiber-optic cable, with just the final leg of the attack being delegated to an AI targeting system.

So, ethics aside, why does drone warfare still depend on human pilots?

Distinguishing enemy targets from friendly assets on the battlefield still requires humans. The main problem is that battlefields are intrinsically adversarial environments: an autonomous drone needs to avoid friendly fire on its own infrastructure and troops, anticipate pre-positioned counterdrone defenses, deal with camouflage and decoys, and destroy hardened targets like vehicles and infrastructure. This is especially complicated if you need drones to autonomously work together to accomplish an objective, such as by having specialized units target defenses to allow others through. For the moment, only humans have the skills to distinguish a camouflaged ally from an enemy unit, or to exploit the underbelly of an armored vehicle.

Designing drones for indiscriminate mass destruction is easier than for precise battlefield use. Unfortunately, adversarial target selection is not a meaningful barrier for applying drones to mass terrorism. Destroying an armored vehicle needs the skill to reason about and single out its weak points; but, to kill an unarmored human, a drone need only make contact with them and deploy an explosive or poison needle. The targeting requirements and level of autonomy needed to indiscriminately massacre civilians, in other words, are much lower than what you’d need to selectively destroy hardened targets on a battlefield. In the words of Ukraine’s Azov brigade, “If you don’t care about civilians, you can simply hit any target that moves.”

Drones can already navigate indoor environments and track humans. Aside from requiring guidance for target selection, autonomous drones also need the ability to navigate. Urban environments are cluttered and leave room for targets to shelter indoors, so drones must infiltrate and sweep through them to be maximally lethal. Autonomous navigation of this caliber already exists: since as far back as 2022, drones have been capable of mapping and tracking indoor environments to find humans, a skill used to locate hostages and scan through tunnels for enemy soldiers. These kinds of drones typically cost tens of thousands of dollars, but they have an expensive use case: infiltrating a GPS-denied location, then escaping to broadcast information. If you do not need the drone to survive and report back, and if you do not care if your drone can tell whether someone is surrendering or not, then you do not need expensive sensors or plenty of onboard compute for decision-making—just the bare minimum to identify a target as human and fly at them.

Disposable drones that can autonomously attack civilians may soon be relatively cheap. For comparison, a last-mile module, an upgrade chip that lets FPV drones visually hunt down targets when they lose connection, is about $500. One-way autonomous search, target selection, and mapping, at least for this anti-civilian use case, would likely be similarly inexpensive—already, visual and laser mapping systems have been demonstrated, in principle, that can work their way around a room and track humans on a few hundred dollars of hardware. If a military system that navigates to the entrance and then plans an indiscriminate suicide mission inside is not already achievable for just thousands of dollars, it will be in a matter of years.

Of course, most countries do not have the motivation to build these kinds of systems and drive down their unit economics. Discrimination, ethical or otherwise, is useful on the battlefield. If the costs of autonomous targeting keep falling or AI guidance improves, however, states might be tempted to start employing indiscriminate drones as a means of deterrence, or as a cheap way to enable terrorist proxies.

Future Methods of Weaponization

Given these capabilities, how could these drones be weaponized and delivered in practice? The main limiter on these kinds of small drones is energy: a modern 30g micro drone like a Black Hornet can fly for about half an hour before needing to recharge, while a bigger FPV carrying an explosive payload will usually last only 15 minutes. However, there are plenty of ways to stretch this energy budget further for the purpose of mass destruction, even without better battery technology.

Lethal payloads could be much smaller. Today, FPV units usually carry about a kilo of explosives, because they need to be flexible enough to target vehicles and defensive infrastructure as well as enemy soldiers. If the goal is to break windows and kill humans, however, even just 20g of frag explosives is enough at several meters of distance, with much less needed for a lethal wound at near-contact. Alternatively, something as simple as a spring-loaded needle, coated with a microgram quantity of a poison like botulinum toxin or a nerve agent, would be immediately lethal on contact.

Energy expenditure on delivery and flight could be significantly reduced. Rather than have the drones travel constantly under their own power, it is much more efficient to carry them into position with a drone mothership or even the hull of a missile. Once the drones are released into the air, they then need to navigate to a building without expending much power. A simple way to do this is to give the drone glide wings, letting it drift forward for most of its flight rather than loiter directly. This technique is already used in Ukraine to stretch the range of basic FPVs over 40 miles beyond the front line.

Drones could perch and idle, rather than hovering, while waiting for targets. Finally, and most importantly, the drones would be ambush hunters. Rather than loiter in the air, it is much more efficient to perch and idle under cover, running a milliwatt acoustic and visual sensor every few seconds until a target is detected. Although drones would still need to expend the energy to infiltrate a building, once inside, they could afford to act like improvised landmines for days or weeks until their batteries finally die.

What an Urban Drone Attack Might Look Like

To appreciate the implications of these capabilities, it helps to outline what a mass urban drone attack would actually look like.

At the outset, drones are delivered near the city through a large mothership, which independently stores them in a cargo hold. Depending on whether the mothership is itself a larger drone, a plane, or the warhead of a missile, it could feasibly deliver anywhere from hundreds to tens of thousands of drones at once. Alternatively, the drones could be smuggled in through a pre-positioned shipping container, which would then launch its contents from a nearby port or logistics yard. Either way, a large number of drones are then scattered above the city at a low altitude or are dispersed near street level.

How many drones? Assuming that the drones are arranged like capsuled quadcopters, they can be packed in extremely efficiently. Using the Ninox-40 system as an example, storage counts would be in the range of:

Once released, the drones glide down haphazardly, aiming to land in regions not already populated by other units. Some drift directly onto public crowds and attack their targets right away. Others do not land near a direct target, switching their focus to look for entrances, such as doorways, windows, and tunnels. If the attacker is particularly sophisticated, these drones might be accompanied by a handful of larger drones carrying breaching charges, designed to create more openings for the main force. These drones then begin drifting through the building interior for a few minutes, looking for victims. If they cannot find a new target or are unable to find a route inside, they move to a dark corner or roadside and begin passively idling.

Many civilians would die in the initial attack. But the aftermath would be much worse. With the drones saturating the city, there would be no opportunity for any survivor to leave their barricade and seek help, and no way for resources and relief units to flow back in. Anyone in need of food or basic medical attention would be unreachable for weeks, during which the death toll would continue to mount. In effect, the entire city would be transformed into something akin to the Ukrainian front line today, with the omnipresent threat of assassination forcing the surviving humans to slowly starve in isolated foxholes, without any chance of easy respite or rescue. Most people would not have stashed food ahead of time, or armed themselves with anti-drone weapons, or sealed off every corner of their apartment with netting.

Defenses Against Drone WMDs May Be Inadequate

Any improvements in drone technology will still need to contend with counterdrone defenses. It is precisely because drones are so threatening that states have invested heavily in tools like directed energy weapons and interceptor drones to counter them. If drones become even more strategically dominant, then we will surely see correspondingly greater counterdrone efforts. So why should we expect this arms race to resolve in favor of the drones, rather than their countermeasures?

Future drones will be less vulnerable to radio-frequency jamming. To appreciate the limitations of counterdrone technology, we can look at the difficulty states are already experiencing in their efforts to counter fiber-optic FPV units. These drones work by unspooling a thin fiber-optic cable behind them, letting a human pilot them directly for up to 40 kilometers without having to worry about GPS or input jamming.

The reason these drones are so effective is that they’re naturally resilient against common counterdrone techniques. Throughout the Ukraine war, the most important anti-drone tool has been radio-frequency jamming. As long as a human pilot is selecting targets and telling the drone where to go, or as long as the drone depends on GPS coordinates to navigate, overwhelming or spoofing those broadcasts with a countersignal will cause the drones to fly harmlessly off course. With a fiber-optic drone, all the piloting happens through a direct data link, so this kind of countersignal is ineffective. The same is true of any autonomous drone—as long as all the decision making is processed on board, there’s no human input to jam or spoof in the first place.

Interception is asymmetrically difficult against small, stealthy drones. In cases where it’s difficult to achieve an electronic soft kill on a drone, the backup option is to physically destroy it with a kinetic interceptor. The reason this is a backup is that it’s expensive and prone to blind spots. Small FPVs are cheap and agile enough that it’s easy to spend much more to down them than they’re worth. This is why direct kinetic interception is usually reserved for expensive drones with fixed flight paths (like Shaheds) and kept as an option of last resort for small drones.

There is also the problem of terrain blindness: if you cannot see a drone, you cannot shoot it down. Modern LSS (low, slow, small) drones are already so tiny that they are difficult for radar to distinguish from birds, trees, and ground clutter, allowing operators to fly them near the treeline until they are too close to reliably intercept. This is especially problematic in an urban environment, where there are many places to hide and many opportunities for collateral damage.

Physical barriers and EMP weapons will struggle to reach sufficient coverage in cities. Instead, the most effective tools against autonomous drones will be structural barriers and directed energy weapons. One of the most visible effects of the war on Ukrainian infrastructure is the miles-long corridor of nets that cover roads and even cities, without which transports would be constantly exposed to drone strikes. But nets are not hard barriers: if the initial carrier punches through, if a few specialized units carry breaching tools, or if the drones are released from below, there is little to stop them from flying unimpeded and besieging the streets beneath.

To actually destroy the drones efficiently, the best candidate is a directed energy weapon, particularly an electromagnetic pulse (EMP) device. It creates an electric pulse powerful enough to short-circuit any electronics in range, physically destroying the drone controllers and motors. Unfortunately, cities are far from ideal places to deploy such weapons; building materials is reasonably effective at shielding against an electric pulse, which means that, if the EMP does not catch the drones in the initial sortie, lack of a clean sight lines will significantly weaken this countermeasure against drones dispersed throughout a city.

In other words, these defenses can be locally effective but struggle to get sufficient coverage, especially over an active residential area. They are also, of course, proactive defenses: they have to be installed ahead of time in order to have any defensive effect, which gives the attacker time to assess and plan around them.

Strategic Applications for Rogue Actors

In summary, small lethal autonomous weapons will be very difficult to defend against, especially in the context of securing large urban environments. This leaves the question of which actors would want to use them, and to what end.

Advanced drones would be more useful for rogue states than for superpowers. The first countries to acquire fully autonomous drones will be those that have a precision manufacturing base and frontier AIs capable of helping with military R&D: namely, the US and China. However, autonomous drones are only marginally useful for the existing great powers: a new option to further deter an invasion of Taiwan, a cheaper way to conduct remote assassinations, and a way to maintain conventional parity with rivals’ own drone mass. Rather than meaningfully changing the balance of power between nuclear states, near-term autonomous drones will likely be most useful for rogue actors seeking new weapons of mass destruction to enhance deterrence. In this regard, indiscriminate autonomous drone swarms possess many advantages.

Drones are inherently simple, which makes it very difficult to control their proliferation. A central reason why modern military drones are so cheap and widespread is their simple design, combined with the commodification of key components like memory and compute. As we have seen with semiconductor sanctions on Russia, the underlying materials are too accessible in ordinary consumer supply chains to easily deny mass production. Moreover, even if a specific country could be cordoned off from general drone production, Russia and China have proved willing to help export their military designs to allies. Even terrorist groups might be able to build, or at least acquire, large quantities of drones for urban attacks, either through covert smuggling or state sponsorship.

Drones can be used conventionally, not just for WMDs, making restrictions hard to enforce. To the extent that states have tried to impose restrictions on drone manufacturing or acquisition, they have largely failed to do so. This is partly because there is not a clear point of intervention in drone development: unlike other WMDs, whose primary purposes are terror and leverage, drones have legitimate civilian and conventional military uses. As a result, international organizations like the UN have done little more than condemn the principle of autonomous weapons, without yet addressing basic questions like the definition of autonomy.

Drone swarms are a far more precise, controllable deterrent than other nonnuclear WMDs. Chemical weapons, although useful for terror, are difficult to widely disperse and threaten entire cities with. This combination of extreme fear and limited destruction thus invites the risk of extreme escalation in retaliation, making them poor deterrents. On the opposite end of the spectrum, bioweapons are simply too destructive, symmetrically threatening those who deploy them, as well as too invisible and delayed to create an immediate effect. Massive drone swarms, however, could be used to reliably besiege an entire city while remaining contained within it.

Autonomous weapons are fundamentally hard to stop, particularly when aimed at civilians. Even if states are able to secure individual targets, such as military bases and political offices, securing the whole of society such that there are no soft targets for advanced drones would be enormously challenging. Even aside from their sheer size, cities are difficult to cover because they are both open enough to leave people exposed while moving and dense enough that interceptors will usually lack a clear line of sight, risking collateral damage. If this technology were to proliferate widely, the future might be one of constant and extreme geopolitical tension, where even minor military powers are tempted to assemble large swarms of murderous drones as deterrents.

Taking Drone WMDs Seriously

By all appearances, the kind of indiscriminate weapon described above is not far off. Autonomous drones that blindly hunt down humans and besiege cities, if they do not already exist, are held back more by ethics and military opportunity cost than any fundamental engineering problems.

But taking this threat seriously also means looking further ahead. There’s no reason to imagine that dumb, flying landmines are as far as drone technology will progress. Drones the size of mosquitoes, drones as cheap as bullets, drones so numerous their swarms blot out the sky, drones that never leave, sitting on your power lines and in the sun.

These are not only possible but inevitable: if all that happens is just the normal grinding of drone engineering and mass production, cheap swarms of thousands, or even millions, of killer drones will eventually be universally available. Whether this happens in 5, 10, or 15 years is much less important than whether we are prepared to deal with it when it does. Some policies, like nonproliferation and defensive investment, can be effective only while the threat is still unrealized. To implement these policies for drones, and for all other future military technologies, we have to take what will be possible tomorrow seriously and plan for it today.

Thanks to Dan Hendrycks for advising on the premise of this piece.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Felix Choussat researches the geopolitics of advanced AI at the Center for AI Safety (CAIS), focusing on Sino-US competition and emerging military technology. Prior to his current role at CAIS, he was a governance fellow through the ML Alignment and Theory Scholars (MATS) and Pivotal Research programs, where he worked on modeling the proliferation of WMD-capable systems and compute-based AI deterrence. He holds a dual degree in international relations and history studies from Vanderbilt University.

In September 2025, the state of California made an AI assistant, Poppy, generally available to employees, to help them “explore AI’s productivity benefits.” The following month, the North Dakota Legislative Council deployed AI to assist with summarizing bills. And, in March 2026, the Los Angeles Superior Court partnered with an AI company to provide support across several key functions, including “case information, summarization, research, analysis and drafting assistance,” as well as case management.

In the near future, it is likely that more US states and the federal government will direct the technology toward even more sensitive and significant tasks. AI may be used to adjudicate disputes, accelerate law-enforcement activities, and aid military operations to an even greater extent than it does today. As AI use cases are more regularly documented, scrutinized, fine-tuned, and optimized, people may actually demand increased AI adoption among government actors, even in sensitive domains. That raises the question: which models should be used?

This question has no simple answer. With other technologies, the choice of a specific model might largely come down to trade-offs between cost and performance. When it comes to AI, however, different frontier models may show significant differences in “character” or appear to align more closely with a particular political worldview. In choosing AI models for government operations, democratic states must therefore consider how best to represent the will of the people.

Currently, the federal government neither measures how a deployed model’s reasoning compares with the public’s nor asks whether divergence between the two is justified. In this piece, we make the case for a body that would answer the first question and equip officials to answer the second. It would keep the reasoning behind government AI open to public view and open to challenge when the people it serves see fit.

How Model Character Could Influence Policy

Model character is a complex property formed through numerous factors. The character of each model—how it tends to respond to certain prompts and perform certain tasks—is a product of many specific decisions that are presently made by a small number of AI engineers working in a handful of frontier labs in an “informal” and evolving process. Tweaks to the training data, the algorithms used to train and fine-tune the model, and company policy related to the model’s banned actions all shape character, among many other factors.

Frontier developers have different approaches to shaping model character. If you read Claude’s Constitution and OpenAI’s Model Spec—the values that Anthropic and OpenAI, respectively, hope to infuse into their chatbots—you’ll see that, while the two labs outline some similar principles for their respective models, there are key differences. For instance, Claude’s Constitution outlines an ideal character for the model—namely, being a “good, wise, and virtuous agent,” whereas OpenAI’s Model Spec provides more explicit directions around what behaviors to pursue or to avoid and how to specifically adhere to a hierarchy of instructions.

An AI model’s character could affect policy in both sudden and gradual ways. As AI systems are increasingly integrated into governmental decisions, the selection of one model over another might alter how presidents respond to crises, how lawmakers evaluate policy, and how judges draft opinions. For instance, depending on whether a congressional office relies on Claude, Grok, or ChatGPT, it may dismiss or fail to identify certain policy options.

In addition to such discrete moments of AI influence, serial exposure to a particular model’s assumptions, framing choices, and preferred forms of reasoning could gradually shape how government officials understand policy problems and evaluate trade-offs. Our fear recalls the proverbial frog in a pot of slowly boiling water: the choice of one model over another would alter many small decisions that add up to a major redirection in policy, regulation, and norms. Clearly, there is a need for a sober analysis of these risks.

Subscribe now

Frontier AI development is currently too opaque to understand how character decisions are made. How should the public and, by extension, government stakeholders proceed with such a weighty task? Should they focus on an audit of the lab’s training data? Should they review the lab’s training process? Should they scrutinize the “constitution” or equivalent document the lab has crafted to shape the model’s character? And, within each of those possible inquiries, how should they rank which model is better than another? We don’t have the answers to those questions—in part because those inquiries are not feasible, given the current level of transparency (or lack thereof) across the labs. Indeed, there is no legal obligation for labs to disclose those core determinants of model character. Whether there should be is a topic best left for another essay.

Few frontier models currently seem to be politically neutral. Notably, agencies are working through some of these questions, but no standardized approach has emerged for a comprehensive procurement evaluation. Pursuant to Executive Order 14319, “Preventing Woke AI in the Federal Government,” agencies are supposed to apply two “Unbiased AI Principles”—truth-seeking and ideological neutrality—when selecting models. Yet, according to recent testing by The Washington Post, Gemini 3.1 Pro and Claude Opus 4.8 are the only leading models that appear to provide ideologically neutral answers to policy questions at least a majority of the time; OpenAI’s ChatGPT-5.5, in stark contrast, provides left-leaning answers in the vast majority of instances. How agencies are supposed to weigh these differences, and at which point a model’s tendency to provide skewed responses becomes a bar to its use, remains unclear.

Introducing Public Reasoning Fidelity

One approach to selecting AI models in line with democratic principles would be to choose the models that most closely mirror public reasoning in various decision-making scenarios.

Public reasoning fidelity aims to identify models that best represent the public worldview. Our new framework, public reasoning fidelity (PRF), involves a process of asking a representative sample of the public to review hypothetical scenarios, ranging from whether to declare war to how to resolve a complex legal case. Models would then be tested on those same scenarios. Using this approach, the model whose outcomes and reasoning most closely resemble those of the public would be given a significant preference in procurement decisions and usage policies. At the very least, there would be human-generated benchmarks for the behavior of a model within a particular use case.

A model’s reasoning—not just its ultimate decision—should track that of the public. A model that reaches the same bottom-line answer as a representative public panel but does so for reasons the public rejects should not receive the same score as a model that mirrors both the public’s resolution and its path to that resolution. In government, the rationale behind a decision may matter as much as the outcome itself, because explanations shape precedent, accountability, and public trust. A judge, legislator, or agency official is not merely choosing between results in most contexts. Each is relying on an explanation that may shape how future questions are framed, which facts are treated as relevant, and which trade-offs are placed at the center of public decision-making.

PRF could select different AI models in different domains. Ideally, the PRF process would occur in specific domains. For instance, the selection of the model used by law enforcement should be grounded in the model’s PRF score on police-related hypothetical scenarios, and it should be conducted separately from model selection for military use cases. This would reduce the odds of PRF scores being too broad to be meaningful to procurement officers looking for a model that is likely to be deployed in specific domains in which the public may have unique preferences and rationales.

The promise of this approach is that it gives public institutions a structured, human-generated benchmark in relevant domains where today they tend to rely on vendor claims and internal testing. PRF is not a silver bullet, though. A quick overview of its potential pitfalls reveals why adopting PRF would require additional safeguards.

One pitfall of PRF is that models could match public judgment to a fault. Aligning with the public’s reasoning may satisfy concerns about a model drifting from how the people approach an issue, but it might also mean the model leans on questionable policy analysis. As explained, PRF rewards similarity, not quality. A model earns a high score by reasoning as the public reasons, which is sometimes but not always the same as reasoning well.

On questions where the considered public judgment is mistaken, or simply less informed than the record allows, a model that tracks the public will be rewarded precisely for its errors, and a model that reasons its way to a better answer will be penalized for departing from the crowd. This is Goodhart’s law in its familiar form. Once fidelity to public reasoning becomes a procurement target, labs will optimize for it and might avoid alternative development practices that help models reason better than the public. In short, the risk is a kind of policy-analysis sycophancy, a failure mode the labs already struggle to suppress.

The PRF process must be designed to avoid collapsing into policy homogeneity. The danger deepens when PRF informs model selection by government officials. If agency staff or lawmakers subtly adopt the reasoning embodied by their model of choice, and if PRF then selects the model whose reasoning most closely matches the public’s, the instrument may end up producing excessively homogeneous policy proposals. Our deliberative processes work best when they allow for nuanced consideration of a wide range of perspectives. The PRF process must be designed to prevent it from undercutting that characteristic.

On many salient questions, there is no single public or reasoning approach to be faithful to. On issues including abortion, firearms, immigration, and election administration, the public holds not one considered judgment but two or more, sorted sharply by party. Here, any single PRF target is a fiction that averages into a position almost no citizen holds. For these polarized domains, the sensible move is to stop asking whether a model matches the median and start asking whether it can represent the competing lines of reasoning fairly rather than collapsing them into one.

That reframing connects PRF to the neutrality criterion that the federal government has already gestured at in Executive Order 14319. A model that can articulate the strongest version of each side, and does not systematically resolve contested value questions toward one pole, is closer to what “ideological neutrality” is reaching for than a model that happens to match a manufactured center.

In more technical domains, experts could communicate the facts to the representative public panel. The hardest problem is likely to be generalization. A representative panel probably has intuitions worth eliciting on whether to declare war or how to resolve a vivid legal dispute. It likely has far less to offer on the capital adequacy of regional banks, the ozone standard under the Clean Air Act, or the fiduciary rules governing retirement plans.

One fix is to split the exercise into the two tasks it actually requires. A domain committee, explained in more detail below, builds the record. Drawing on the experts among its members, the committee curates the facts, translates the jargon, and lays out the competing arguments. The judgment still comes from an informed lay panel that works through that record, much as deliberative polls and citizens’ assemblies have done on technical questions, from electoral reform in British Columbia to constitutional change in Ireland. The committee informs the public but does not stand in for it. PRF then measures whether a model weighs the trade-offs as an informed public would, once the record is set.

In the most specialized fields, a lay panel will track whichever expert framing proves most persuasive, so PRF there measures fidelity to the committee’s framing as much as to the public’s reasoning. The upshot: in technical domains, a PRF score is only as good as the committee that built the record. Avoiding these pitfalls will require robust, carefully considered oversight of the entire PRF process. This is why the institutional design that follows is of critical importance to PRF’s odds of success.

Designing an Institution to Evaluate PRF

A process as consequential as PRF would need an institutional home. At the federal level, Congress could create a standing “Commission on Public AI Use,” housed within the Center for AI Standards and Innovation, which is home to the leading AI experts within the federal government. This commission could oversee the curation of hypotheticals, the selection of representative public panels, and the comparison between public responses and model responses (note that states should also explore the creation of such bodies—the focus of this essay is at the national level). These scenarios might subsequently be shared with subnational agencies.

The commission could rely on domain-specific committees to develop hypotheticals. Within the commission, a judicial-use committee could include former judges, legal scholars, practicing attorneys, court administrators, technologists, and members of the public. A law-enforcement committee could include former prosecutors, defense attorneys, civil rights lawyers, police officials, local-government representatives, and community members. Importantly, each committee would develop sealed hypotheticals designed to test the kinds of questions that may arise in that domain. Of course, those hypotheticals would not be disclosed until the testing period, to reduce the risk that labs train to the test.

The public panels, not the committees, are the basis of the benchmark. Each committee builds the record and writes the questions. But, to be abundantly clear, its panel renders the judgment that the model is scored against. “Representative” here refers to panel members being drawn by lot and stratified to mirror the domain’s relevant population, on the deliberative-polling model described above. For a model used in immigration adjudication, for example, the panel should reflect the demographics of the communities that appear before immigration courts. Panel members are not appointed, and they may not have their judgment usurped by a committee of experts.

Each representative panel would review common factual records and competing arguments before producing their own outcomes and reasoning. AI models would be given the same materials. The committee would then compare the models to its public panel across two dimensions: whether the model reached a similar outcome and whether its reasoning reflected the same concerns, priorities, and limiting principles.

The commission appointment process should be designed to avoid political capture. What stops a new administration from reshaping the exercise to fit its preferred style of governance? The panel resists capture on its own terms because no one can pack a lottery. Each cycle draws a fresh random sample, so partisanship enters the panel only in the proportion it holds in the population, and it enters each time anew. The commission, however, whose members are appointed, is more exposed to political machinations. Congress should armor it as it armored the US Sentencing Commission, which caps single-party membership at a bare majority of seats, sets staggered six-year terms that outlast any single presidency, and permits removal only for cause.

Precedents

While setting up new oversight bodies and processes designed to operate well over the long term is a significant challenge, it has been done successfully before.

The proposed institutional design would not be entirely novel. The US Sentencing Commission referenced above offers a useful analogy. It operates in a highly sensitive domain, translates legal and policy judgments into structured guidance, and attempts to promote consistency without eliminating judgment. The commission establishes sentencing policies and practices for the federal criminal justice system. Congress took due care to place expertise within the commission—including designated slots for former judges—while still leaving tremendous discretion to the judges tasked with applying the commission’s recommendations for sentencing lengths.

The Administrative Conference of the United States serves as another example. ACUS does not run agencies, but it studies administrative practice and issues recommendations designed to improve fairness, efficiency, and accountability across government.

A Commission on Public AI Use would play a similar role for AI adoption. Importantly, it would not replace elected officials, judges, agency heads, or procurement officers. It would give them the results of a benchmark developed in a participatory process. Rather than relying solely on vendor claims, internal testing, or the informal preferences of government employees, public institutions would have access to a structured assessment of how competing models reason through hard cases compared with the considered judgment of the people those institutions serve.

Some will view this process as premature or overly cumbersome. That critique overlooks a more immediate reality: AI models are already shaping how government actors make consequential decisions, yet the public has almost no meaningful role in overseeing how those systems are selected or evaluated. While the PRF mechanism may not be perfect, an oversight system must be developed. Absent such oversight, there’s a risk of civil servants, agency heads, and elected officials trying to blame poor decisions on AI. Those excuses would have far less weight if the public and policymakers alike knew more about how models operate in particular contexts.

PRF does not hand all decisions to the public, but it allows public scrutiny of government AI tools. One obvious concern with this proposal is that public opinion is not synonymous with sound governance. Presidents, judges, legislators, and agency officials routinely make decisions that depart from majority sentiment because they are bound by constitutional constraints, institutional obligations, classified information, or technical expertise unavailable to the general public. As noted above, a model that perfectly mirrors public sentiment may therefore still be poorly suited for certain governmental functions.

That concern should shape how PRF is understood. The purpose of PRF is not to hand public polling the reins of government decision-making. Nor is it to create a plebiscitary mechanism for selecting AI systems. The point is narrower and more practical: public institutions should know whether the models they rely on consistently reason through difficult questions in ways that diverge from the public.

At present, that divergence is almost entirely invisible. Agencies, courts, and legislatures may adopt tools whose assumptions, value judgments, and interpretive tendencies subtly shape official decision-making, without any meaningful public scrutiny. PRF would help surface those tendencies. In some cases, decision-makers may conclude that a model’s divergence from public reasoning is justified by legal doctrine, technical realities, or institutional constraints. In others, that divergence may raise concerns about legitimacy, accountability, or democratic responsiveness. Either way, the divergence itself should not remain hidden from the public.

The Public Should Choose Public AI Values

Government adoption of AI should not proceed as though model selection is ordinary software procurement. When public institutions rely on systems that reason, rank values, frame trade-offs, and influence official judgment, the public has a legitimate interest in knowing how those systems think through hard cases. PRF would not answer every question raised by government AI use and may not be useful in certain domains, but it would make one question harder to avoid: whether the models acting in the public’s name reason in ways the public can recognize, evaluate, and contest. That is the minimum a democratic government should demand before allowing private model choices to become public governing defaults.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Kevin Frazier is the Inaugural AI Innovation and Law Fellow at Texas Law.

Andrew W. Reddie is an Associate Research Professor at the University of California, Berkeley’s Goldman School of Public Policy, and Founder and Faculty Director of the Berkeley Risk and Security Lab.

How should governments regulate the most advanced AI systems? One possible answer is that they should not. Libertarian-minded writers have made the case for a culture of “permissionless innovation” for AI development, in which the role of government would be limited to enforcing existing laws and facilitating industry self-regulation with “soft law” tools such as voluntary standard-setting. Another possibility, more in vogue across the aisle and the pond, would invoke the precautionary principle, using heavy-handed regulation to restrict the development of frontier AI models until developers can adequately prove that their systems are safe.

I think there’s a better approach. In a new essay, my co-author Christoph Winter and I make the case for a governance strategy that we call “radical optionality.” The idea is simple: governments should avoid over-regulation in the short term while building up the institutional capacity needed to competently regulate extremely advanced or “transformative” future AI systems when and if these systems come into existence. The point of this approach is to maximize optionality by providing our institutions with tools that can be used to respond to a wide range of foreseen or unforeseen future developments.

In this piece, we explain the rationale behind maximizing optionality while AI’s future impacts remain uncertain. We also outline some example measures that governments can take and explain how this approach dovetails with other proposals for AI governance.

Addressing AI Risks Under Uncertainty

Leading AI researchers in academia and industry have claimed that advances in AI capabilities may soon produce “AGI,” “artificial superintelligence,” “powerful AI,” or some similar term. If you are certain that these statements are hype, and that such advanced AI systems will not arrive during our lifetimes, I won’t try to convince you otherwise; enough ink has been spilled on the subject that I’m not optimistic about my ability to contribute anything new. But if you think there is even a small chance that these predictions materialize, or if you find them at all credible, we think that the argument for radical optionality is overwhelmingly strong. The argument goes as follows.

AI’s future impacts are highly uncertain. Assume that there is some possibility of transformative AI systems being invented within the next, say, 15 years or so. Most of us are extremely uncertain about exactly how and when this will happen, what the characteristics and tendencies of these systems will be, what benefits they will offer society, and what risks to public safety and national security they will create. Under some assumptions, these systems will be mostly harmless and highly beneficial, because the companies creating them will have incentives to make them safe and broadly aligned with human preferences. Under other assumptions, these systems will be dangerous and difficult to control—perhaps even capable of causing human extinction if the right guardrails are not put in place. Maybe securitization is inevitable and the U.S. government will soon spring into action and develop these systems behind closed doors as part of a clandestine military project. Alternatively, perhaps development will happen in a decentralized and democratic way.

Subscribe now

The question of regulation seems to present a tradeoff between innovation and security. Debating which of the scenarios described is most realistic can be valuable, but ultimately only ideologues claim to be certain about the future course of a poorly understood emerging technology. The rest of us have to make important decisions about what to do while acknowledging substantial uncertainty. On the one hand, restrictively regulating AI companies would slow down innovation, potentially depriving society of some of the benefits of technological progress. On the other hand, it is possible that well-designed regulations could mitigate the real risks that AI systems pose, both now and in the future. How should we think about this tradeoff between innovation and security?

Measures that maximize optionality can improve security without hindering innovation. We think that this framing misses an essential point: there are steps governments can take now that would increase security without any significant cost to innovation. At the top of the list are light-touch information-gathering authorities like whistleblower protections, reporting requirements, and transparency mandates. Government agencies thrive on a diet of information; it has been said that “information is the lifeblood of good governance.” Authorities that increase the government’s access to important information about AI risks—and allow the relevant agencies to develop expertise in securely processing and interpreting such information—are foundational building blocks for future governance efforts. Mechanisms for securely and intelligently sharing information within government, and (when appropriate) between governments, are similarly foundational.

Building capacity directly is also important. First and foremost, this means enabling the relevant regulatory bodies to hire and retain elite talent. Meta’s recent hiring spree, featuring 10-figure compensation package offers for top AI talent, is an example of what it looks like when an organization takes the prospect of transformative AI seriously. Governments will likely be unable to compete with the salaries on offer in the private sector, but reforms to processes for government hiring and contracting of AI talent are nevertheless needed in both the U.S. and the EU. The early successes of the UK’s AI Security Institute, which receives 10 times the funding of its U.S. counterpart despite the UK’s relatively modest GDP and industry relevance, demonstrates the importance of cultivating talent in government.

The full-length essay discusses a number of other optionality-increasing policy decisions, such as incentivizing lab security, avoiding premature and overbroad preemption of state laws, and building out an ecosystem for model assessments and evaluations. But the important thing is to recognize that security and innovation are not conflicting priorities, because there are ways to increase optionality without creating any significant barriers to technological progress.

Supporting Both Security and Innovation

Radical optionality is compatible with other proposals for AI governance. Radical optionality is by no means the first AI governance framework to recognize that governments have an important role to play while also acknowledging that overly restrictive regulation could hinder innovation. Dean Ball and Gillian Hadfield and Jack Clark have proposed sophisticated private governance regimes in which the government would certify an ecosystem of private regulators competing to offer efficient and nimble regulatory services to companies on an opt-in basis. Gabriel Weil has argued that a well-designed tort liability regime, featuring insurance requirements, punitive damages, and strict liability for certain harms, could force AI companies to internalize any risks generated by their products. And Cary Coglianese has advocated for a system of management-based regulation, requiring AI companies to take risk mitigation measures but allowing them broad discretion over what measures to implement and how. I view these proposals as consistent and compatible with radical optionality; tort liability, management-based regulation, and private governance mechanisms are valuable tools for maintaining and increasing optionality.

Under most worldviews, radical optionality is preferable to the status quo. Of course, not everyone will agree that an optionality-maximizing approach is wise or sufficient. If you confidently believe that the safety benefits of a restrictive AI regulatory regime would outweigh the costs to society of slowing innovation, it is reasonable to suggest that anticipatory regulation is needed. From this perspective, radical optionality does not go far enough, but it would still be preferable to the status quo. On the other hand, from a libertarian perspective, building up government capacity to regulate and promising that it will not be used prematurely might look a lot like giving the government a hammer and promising that agencies will not start hallucinating nails. I can’t promise that there is no chance of new authorities being abused, or that everyone will agree on when dual-use AI systems have become so advanced that regulating them is a national security imperative. But I do expect building government capacity to benefit AI companies as well as the public in the long term. If rapid progress in AI capabilities research gives rise to a surge in public demand for regulation at some point in the future, as some writers have predicted, companies might prefer for the government to have the option of regulating in a competent, targeted manner.

At my organization, the Institute for Law & AI, we spend a lot of time thinking about how advanced AI systems should be governed in the present and in the future. Radical optionality is a sort of organizing principle and guiding philosophy for that research and consulting work. When deciding what projects to work on, what bills to offer feedback on, and what policies to push for, the question of what approach will maximize optionality is typically one of the foremost considerations. In publishing this paper, I hope to convince at least a few people to adopt this framing, to recognize the importance of optionality, and to begin viewing security and innovation as compatible rather than conflicting priorities.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Charlie Bullock is a Senior Research Fellow at the Institute for Law & AI. He advises state and federal policy makers on AI governance topics and publishes research on legal questions with significant practical relevance to U.S. AI policy. His recent research examines issues including federal preemption of state AI laws, federal and state AI whistleblower protection legislation, and the likely consequences of the end of Chevron deference for the future of AI regulation. Charlie received his J.D. from Yale Law School, where he was an Editor for the Yale Journal on Regulation.

When officials in Washington warn about losing the AI race to China, the conversation turns quickly to military and economic advantage—and rightly so. Advanced AI will reshape everything from weapons systems to medicine, with massive implications for our geopolitical competitiveness. But beneath the great-power competition on AI lies a moral one. The ethical character of the most transformative technology in generations—one that will mediate an ever-larger share of human experience—will be a byproduct of superpower rivalry. At stake is the future of the relationship between individuals and the state, of privacy and control, of human agency and algorithmic authority.

The fear is not merely that China might build better systems, deploy them more widely, or out-sell American competitors. It is that those systems could carry a tide of new norms shaped by a government that surveils its citizens, suppresses dissent, harbors eugenic ambitions, and treats individual autonomy as a problem. As AI comes to dominate our lives as thoroughly as digital media already has—shaping our health and finances, how our children learn, how we are tracked, even our species’ genetic makeup—this battle for AI’s soul will affect us all intimately.

The contours of that battle are almost always left unexamined, but usually assume one of three forms: some leaders suggest AI’s values will be a winner-takes-all byproduct of the race to technical superiority; others imply a conscious struggle to spread tools and platforms with value systems baked in; still others insist that diplomatic cooperation between AI powers is the only way to bend AI’s ethical arc toward humanity’s benefit. Leaving these three paradigms implicit does everyone a disservice, robbing the United States of both moral clarity and strategic opportunity.

Looking at the history of technological competition, there may be truth to all of these three models. But each implies a different approach to maintaining American leadership while preserving the values we claim to champion. And analyzing them clearly reveals how badly our attention is skewed. Today’s debate fixates on breakaway dominance, blinding policymakers to the more consequential contests over encoded values and strategic diplomacy. A rebalanced approach would require something we currently lack: a clear, affirmative vision of what American AI should be for.

Breakaway Tech Dominance

“AI offers the potential promise of extending American hegemony.”

—Tyler Cowen

Breakaway tech dominance is the default (if often implicit) ambition for many leaders invested in the Sino-American AI competition. The thinking goes that if the US is able to master AI ahead of others, that advantage will translate into a general offset in American power over China, with powerful ripple effects in economics, culture, and politics globally. Because this winner-takes-all vision provides such a clear motivation for forging ahead, it is the one most often invoked by pro-tech voices and government leaders.

Historically, large technological advantages tend to precede hegemonic power. This dominant narrative about AI competition draws heavily on historical analogy. Britain’s industrial revolution produced not just economic advantages but also cultural ascent. British institutions, law, language, and ideas spread across the globe on the strength of steam engines and mechanized looms. And this was not history’s first instance of technological offset, a dynamic that has persisted since before the Assyrians’ mastery of iron metallurgy expanded their influence over rival groups.

Applied to AI, this logic suggests that the first nation to achieve a decisive breakthrough could gain civilizational escape velocity. According to this view, if China masters AI before America does, Beijing’s authoritarian model—surveillance systems, social credit schemes, algorithmic control of information and behavior—would spread globally with irresistible momentum. Given that many experts expect the AI transformation to be comparable in scope to industrialization or the dawn of the Iron Age, such fears are justified. Whether or not there are dramatic power shifts between the United States and China in the century ahead, AI is certain to play an outsized role.

According to this paradigm, the singular priority must be aggressive progress in AI capabilities. The strategic implication of this perspective is obvious: there is nothing so important as moving faster than China in pushing the bounds of AI technology. Additionally, there is little need for the United States to consider how American values relate to its AI strategy, because they are seen as downstream of the technical rivalry. In other words, if the United States establishes a decisive AI lead, its values will organically spread; if China masters the technology first, Beijing’s moral vision will take root globally.

An AI lead sufficient to achieve hegemony is unlikely to appear on either side. The breakaway-dominance framework functions only if there is a defensible breakthrough to be had, which is not necessarily the case. Some predictions of a superintelligence “takeoff”—in which a sufficiently advanced AI system starts to improve itself better and faster than humans could—fit that mold. But despite regular predictions of imminent AGI breakthroughs, even many bullish researchers are increasingly skeptical of such a scenario, making the prospects of a highly dominant and defensible AI hegemon seem unlikely.

US-China competition is also too tight for breakaway dominance to occur. The observable pattern of AI progress in recent years suggests a different path. China has successfully positioned itself as an aggressive fast follower. In frontier models, the most competitive arena of AI competition, Chinese labs tend to trail American counterparts by just months at a fraction of the cost. That is an achievement in itself—and it diminishes the likelihood that the United States will achieve a sustained, decisive advantage. While not impossible, it’s unlikely that we will see either country develop and maintain an AI lead significant enough to extend Chinese or Western values globally for any sustained length of time unchallenged. This winner-takes-all model, despite its implicit prominence in many policy discussions, almost certainly misses the full picture.

Encoded Values

“China is doing everything it can to dominate AI globally, and they will program the AI with Chinese values…. We’ve got to double down and make sure that American values are the values of the world, and that we control this global AI agenda.”

—Former US Senator Kyrsten Sinema (I-AZ)

A second model for looking at the moral stakes of Sino-American AI competition is the spread of encoded values: the ethics that are baked into new technologies, whether deliberately or subconsciously. This dynamic is ancient: Roman aqueducts built republican virtues into stone by distributing water first to public fountains, then to public baths, and only later to private homes. Fast-forward to the present day, when the internet stands out as a technology consciously designed with libertarian principles: decentralized architecture, open protocols, and resistance to central control. The resulting technology reflected those values in its most basic protocols (if only initially).

The same will be even truer of AI, given its unique ability to absorb and instantiate value systems. The protocols and architecture around AI systems may also reflect value decisions, but particular moral visions and preferences can also be directly distilled in today’s AI systems—or rooted out of them.

China has been explicit about its intentions to imbue AI with its own values. Official Chinese government regulations mandate that frontier AI systems must “uphold core socialist values”—that is, they must adhere to the Chinese Communist Party’s totalitarian view of history and morality. Chairman Xi Jinping has already made considerable strides toward that end. Beijing invests tens of billions of dollars annually in building a techno-authoritarian ecosystem of tools, platforms, standards, and norms aligned with state priorities: social stability, party authority, and collective “harmony,” rather than individual autonomy. Its companies are experimenting with novel, AI-powered methods of conducting surveillance, enhancing censorship, and even predicting political dissent before it occurs.

US efforts to impart values into its AI ecosystem have been less concerted. The United States has been far less deliberate than China in developing AI consonant with American values. True, documents like the Biden administration’s “AI Bill of Rights” and companies’ interminable desire to write AI-principles documents at least pay lip service to the idea of aligning emerging AI systems with democratic principles. The clearest example of this might be Anthropic’s approach to “constitutional AI,” which aims to evoke the US Constitution in its model operations. And on balance, American AI companies’ systems pay much greater attention to ethics and safety concerns than their Chinese counterparts do. But while these examples reflect a different culture around the development of AI in the United States, they are often only window dressing, and pale in comparison to the concerted state focus that Beijing exerts on the normative trajectory of China’s tech sector.

In practice, American AI may actually erode American values more than it supports them. Indeed, American companies have historically been indispensable in building out China’s techno-authoritarian ecosystem. Between public discourse-corrupting deepfakes, microtargeted political manipulation, and algorithmic amplification of extreme content, leading US developers are already arguably producing AI tools that weaken democracy more than they strengthen it. The governments of both superpowers are working to reap the efficiency benefits of AI in their state bureaucracies. But whereas China’s regime is laying out a proactive vision for how AI will advance authoritarian control, the United States has been largely reactive in accommodating AI to democracy—waiting for courts to adjudicate how new technologies can or cannot be used according to existing American law.

The US needs a robust vision for democratic AI and the will to disseminate it. Breakaway-AI proponents see AI’s future as a straightforward innovation race with downstream ethical repercussions. By contrast, proponents of AI as a system of encoded values see the future as a struggle over vision and will. To further American values and strengthen democracy, this view would require developing a much clearer vision: a compelling idea of how to use AI—not just by reactively implementing guardrails but by proactively conceptualizing what democratic AI should look like and enable. Will, equally important, is the drive to commercialize and aggressively spread the resulting systems around the world in collaboration with allies, while also preventing domestic companies from working with China in ways that undermine the United States’ vision.

The Trump administration, by focusing largely on will, has seen some gains in diffusing US technology. But the PRC still maintains considerable diffusion advantages—especially in the Global South, where China’s price-point advantages and shared development conditions give it an edge in building out AI infrastructure for developing nations. A values-driven AI vision, as described above, remains comparatively underdeveloped on the American side: Washington lacks an inspiring, affirmative narrative about what democratic AI enables for its citizens that authoritarian AI cannot.

The US vision deficit has downstream effects on will. Without a clear, compelling sense of what American companies are building toward, it is impossible to muster the political energy needed to make hard choices. For example, US firms are currently bolstering Beijing’s AI ecosystem in sensitive domains like biotechnology, with little oversight. A stronger vision of American AI would galvanize support for restricting US companies from making such contributions to China’s AI ecosystem.

Emergent Control Regimes

“What Soviet-American nuclear arms control was to world stability since the 1970s, U.S.-Chinese A.I. collaboration to make sure we effectively control these rapidly advancing A.I. systems will be for the stability of tomorrow’s world.”

—New York Times columnist Thomas L. Friedman

The third paradigm for how the moral future of AI hinges on Sino-American competition is that of emergent control regimes—the shared rules and institutions that powers build over time to govern consequential technologies. For policy wonks, this is often the most overlooked—or, more accurately, the most dismissed—avenue for shaping outcomes. In part, it is not taken seriously because those who do raise it tend to do so with flagrant naivete about the weakness of the multilateral system and the political infeasibility of any good-faith agreement between the United States and China. But the idea that international control regimes might emerge over time and prove influential is not far-fetched, particularly in areas such as lethal autonomous weapons and AI-powered human gene editing.

The nuclear era shows that self-interest can drive even rivals to manage powerful technologies together, if imperfectly. Early in the Cold War, the idea that the United States and the Soviet Union could reach an agreement about nuclear weapons seemed fanciful. Nonetheless, both nations came to recognize that countering proliferation was in each nation’s interest, even as they remained locked in an existential nuclear arms struggle. No one in 1945 could have predicted the specific contours of what emerged from the complex, path-dependent interactions among nuclear-armed states with evolving interests over decades: the Nuclear Non-Proliferation Treaty, the International Atomic Energy Agency (IAEA), test-ban agreements, verification mechanisms, and norms around nuclear use. While imperfect, these innovations have unquestionably shaped and constrained the most destructive technology that humanity has yet produced.

Advantages will accrue to whichever power crafts and brands politically feasible international controls. To be sure, there are limits on the degree to which such controls can be planned for, given how contingent they tend to be on changing relations and events. But this is not to say that any control regimes emerging from rapidly advancing AI systems are too contingent to plan for in any way. Such controls are not to be confused with the idealistic proposals—such as Pugwash-style scientist convenings or bilateral red-teaming exchanges—that are characteristic of most current track-two dialogues. Nor are they the feel-good unilateral pronouncements of rosy intentions like the resolution on “the promotion, protection and enjoyment of human rights on the Internet,” passed six times by the UN Human Rights Council since 2012. As with nuclear controls, any diplomatic development of consequence will likely be highly controversial, and will necessarily fall far short of what most peace-loving technologists would like to see.

But imperfect measures can still be strategic. President Eisenhower’s famous “Atoms for Peace” speech in 1953 set the foundation for the IAEA. It also served as a tremendous public relations victory for the United States, projecting America as the responsible superpower, willing to help other countries benefit from peaceful applications of atomic technology. It forced the Soviet Union to compete with the United States in building nuclear reactors for other countries, at a high cost to the Soviets. And subsequent US-Soviet nuclear arms control negotiations did more than help to constrain the risks of nuclear war; they also allowed the United States to pursue advantages in qualitative force capabilities at lower cost, under the auspices of quantitative weapons restrictions.

Compared with China, the US is better positioned to lead emergent control regimes. Technologists have given a great deal of thought to unrealistic controls for theoretical future AI capabilities. Yet little serious thought has gone into diplomacy in those areas where international controls could be made politically feasible, soft-power enhancing, and strategically advantageous. The partial exception is the American-led Political Declaration on Responsible Military Use of Artificial Intelligence and Autonomy, which has made strides toward establishing American leadership in rules around the use of lethal autonomous weapons. This guidance is both strategically beneficial to the United States and resonant with American values. Several other areas show promise for similar interventions, not least the ethically fraught genomic applications of emerging AI-powered biotech and the use of AI in high-risk industries. Here the United States has substantial untapped advantages: a global network of allies, a strong history of effective tech diplomacy, and a brand of AI development unencumbered by China’s dystopian techno-authoritarianism. But these advantages so far have not deterred China’s ambitious efforts to eke out a leading position in global AI governance.

A Rebalanced Approach

These three paradigms of AI competition—breakaway dominance, encoded values, and emergent control regimes—are not mutually exclusive. Some areas of AI may see defensible technological breakthroughs that confer long-term advantages; some will become battlegrounds for embedded values; some will develop controls; and some will combine elements from several of these paradigms. They are also interrelated: if one power successfully embeds its values into a widely adopted technology, it will likely occupy a privileged position in control discussions, for example. The question is not which single model is most accurate, but how to allocate attention and resources across all three, and for which issues.

Yet today’s focus remains mistakenly skewed toward a winner-takes-all narrative, blinding policymakers to more consequential contests on encoded values and creative thinking on strategic diplomacy.

To take one example, while initial US nuclear dominance was essential, it was ultimately short-lived. Many developers of the weapon believed America’s 1945 breakthrough would represent an enduring strategic advantage, similar to how many see the race to superintelligence as today’s single defining competition. But America’s nuclear dominance lasted just four short years. The Manhattan Project was indispensable—the United States’ adversaries getting the bomb first would have been catastrophic. But those banking on sustained dominance were in for a rude awakening. Ultimately, clever nuclear diplomacy contributed more to the United States’ victory over the Soviet Union than breakaway nuclear superiority, which never materialized.

Encoding values in technology requires proactive efforts. To the extent that a technology as broad as AI can be compared to a recent innovation, the best analogue is probably the internet—unfortunately, another cautionary tale. Although American engineers deliberately built the internet with libertarian principles, China has been able to co-opt it through force of will. Today, the Great Firewall and the other tools that the CCP has built into the Chinese internet have transformed a freedom-enhancing technology into history’s most sophisticated instrument of surveillance, censorship, and control.

Beijing is exporting these tools abroad, enabling other autocracies to turn the internet away from its original open-society-enhancing design toward repressive ends. The story might have turned out differently if the United States had engaged in more proactive diplomacy—leveraging its unique influence over the development of China’s internet, instead of just issuing feel-good digital-rights statements. At a minimum, curbing American tech companies’ active support of Chinese technological ambitions would have slowed Beijing’s successful authoritarian conquest of the internet. Indeed, the extent of American support for techno-authoritarian progress casts serious doubt on any assertion that the originating society of a technology will organically imbue that technology with its own values. For the war over embedded values, the internet’s lesson is clear: technology neither establishes nor preserves values passively.

The US cannot rely on technological dominance to ensure that AI furthers American values. The United States must learn from these historical cases quickly. AI-powered Chinese “smart cities” are already spreading across the Global South, bringing with them surveillance architectures designed for authoritarian control. Cheap, CCP-compliant Chinese open-source frontier models are already gaining uptake internationally. Party-aligned research centers are developing AI-powered propaganda and censorship tools with unprecedented sophistication; these will soon be diffused abroad, if they haven’t already. Policy and tech leaders may think that their efforts to simply accelerate American technical progress at the frontier of AI innovation will ensure that American values triumph. However, the more probable outcome could be a world awash in cheap, authoritarian AI that outcompetes slightly more sophisticated American offerings that do little to promote American values—and perhaps even erode them.

To course correct, the US must establish a President’s council or congressional commission on democratic AI. A misguided fixation on a winner-takes-all race for technical superiority, as a proxy for a competition of values, risks missing where the real competition lies. It also misses opportunities to rout China diplomatically, similar to America’s successes in nuclear diplomacy. A better approach to AI must start with developing a clearer moral vision for American AI. The President’s Council on Bioethics, established under former President George W. Bush, offers an effective model of what this could look like: a substantive body bringing diverse perspectives to the highest levels of government to grapple with emerging ethical challenges, producing influential reports that shaped discourse and policy. A comparable council on AI and democratic governance could build the intellectual foundations for techno-democracy that do not yet exist, as the American Enterprise Institute’s Council on AI Ethics is beginning to show.

Armed with a clearer, more compelling moral vision for AI, American technologists and policymakers could be galvanized toward supporting the United States’ competition with China with sharper focus. Such a vision would also provide a basis to more aggressively curb American companies’ substantial aiding and abetting of China’s techno-authoritarian ecosystem. And it could provide a stronger foundation for closer collaboration with indispensable like-minded partners such as India: nations better equipped to compete with China on rolling out price-competitive and context-relevant AI offerings in the Global South.

There is little doubt that the Sino-American battle over AI will have tremendous consequences for the future of humanity. Approaching that contest with greater moral clarity is not just the right thing to do; it is also a strategic imperative.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Bill Drexel is a senior fellow at Hudson Institute. His work focuses on United States–India relations, artificial intelligence competition with China, and technology in American grand strategy. Previously, Mr. Drexel worked on technology and national security at the Center for a New American Security, humanitarian innovation at the United Nations, and on Indo-Pacific affairs at the American Enterprise Institute. Drexel’s field experience includes serving as a rescue boat driver during Libya’s migration crisis, conducting investigative research in the surveillance state of Xinjiang, China, and supporting humanitarian data efforts across wartime Ukraine.

Since the late 1950s, Mutual Assured Destruction (MAD) has served as a limiting factor on great-power conflict. The doctrine holds that, if two opposing nations have nuclear weapons that can survive one another’s initial strike, then the near-certainty of devastating retaliation will deter each side from launching a large-scale nuclear attack. Despite this logic, military planners have long considered the possibility of a counterforce nuclear attack, where a superpower uses nuclear weapons to cripple the nuclear capabilities of its enemy. If such an attack were executed preemptively, as a so-called “first strike,” it could both start and end a great-power conflict in a matter of hours: without retaliatory capacity, the defender would be at the mercy of the aggressor’s remaining nuclear weapons and forced to surrender.

The reason this does not happen is that a truly successful counterforce strike is nearly impossible to pull off: a would-be attacker doesn’t know the locations of all opposing missile launchers and submarines, nor does it have missiles with sufficient precision and speed to destroy opposing missile silos before they could launch retaliatory nuclear warheads. Thus, if provoked by a first strike, the opposing side would likely be able to launch a large-scale nuclear response, and the attacker, unable to counter all enemy missiles, could face devastating losses. The result of a preemptive counterforce strike, in other words, would be mutual assured destruction.

AI could change this dynamic. By the mid-2030s, AI-assisted research and development could reduce the cost required to develop and manufacture military hardware by an order of magnitude. Such lower costs could enable a nation with an AI lead to quickly and cheaply build military infrastructure projects, making nuclear counterforce strikes a realistic possibility. Nations with weaker AI capabilities would struggle to quickly build the countermeasures needed to retain a credible nuclear deterrent.

This article will primarily be a technical assessment of how AI could undermine nuclear deterrence, although I’ll lightly touch on a few political aspects. For a longer and more in-depth assessment of the factors discussed here, consider taking a look at an earlier piece I wrote.

Why MAD Has Worked So Far

In any preemptive counterforce strike scenario, the attacker is at a large disadvantage compared with the defender, because they must destroy all of the defender’s nuclear weapons. To execute a counterforce strike, the attacker must achieve all the following requirements simultaneously:

Suppress launch on warning. Many nuclear-armed nations have a policy of launch on warning—launching a retaliatory strike based on sensor data, before a nuclear weapon has been confirmed to land on their territory. The attacker must either use fast-arriving weapons or otherwise disable launch on warning.

Locate and destroy nuclear submarines. Nuclear submarines, while extremely stealthy, are vulnerable once detected. Each nuclear submarine can carry and launch hundreds of warheads. The attacker must accurately track their locations in real time and destroy them all within minutes.

Locate and destroy mobile launchers. China and Russia (although notably not the US, the UK, or France) each field a set of mobile missile launchers that can be dispersed during times of high alert. These mobile launchers are extremely difficult to locate for long enough to successfully strike. However, if accurate and up-to-date position data can be provided, they are easy to destroy.

Destroy all silos. Hardened silos have known positions, but they require almost a direct hit—ideally with a nuclear weapon—to ensure their destruction. They stand out among nuclear launch platforms, as they have the fastest reaction time and the hardest-to-disrupt communications.

Defend against surviving missiles. If the attacking nation has a functioning missile defense system, it may not need to destroy all of an enemy’s nuclear weapons, as its missile defense can handle some leftovers. The stronger a nation’s missile defense, the less thorough its first strike has to be.

Overcome the nuclear taboo. One theory for why nuclear weapons are rarely used is the nuclear taboo. The leaders of both the US and the USSR recognized the gravity of nuclear weapons usage, and had serious humanitarian reservations about starting a nuclear conflict. Even if a counterforce strike were perfect, radioactive nuclear fallout was expected to kill tens of millions in the target country.

The six requirements above have thus far preserved deterrence not because they are physically impossible to meet but because meeting them at the scale required has always been prohibitively expensive.

A useful paradigm to consider in nuclear conflict is the cost-exchange ratio. When one side fields an additional weapon, how much must the other side spend to neutralize it? The concept comes from the Cold War debate over ballistic missile defense. If an attacking nation can build another nuclear missile for $1 million, and the interceptor needed to stop it costs the defending nation $10 million, then missile defense is a losing game: each $1 spent on offense forces a $10 expenditure on defense. This particular cost-exchange ratio explains why a nation launching a counterforce strike would struggle to defend against surviving missiles.

Why AI Could Dramatically Cut Military Infrastructure Costs

AI-assisted military R&D could invert the cost-exchange ratio. If a nation with an AI lead can automate most of the design, systems integration, and production labor that currently makes military infrastructure expensive, that nation’s costs could be significantly reduced while its enemies’ costs stayed the same (since they don’t have the AI advantage). If interceptors cost only $100,000 and missiles still cost $1 million, then it suddenly becomes rational to build more interceptors.

The likelihood of this prediction’s coming true rests on answers to two questions: (1) whether AI can automate most of the intellectual labor, and (2) whether doing so really cuts costs by a margin significant enough to change militaries’ economic calculus. Notably, this forecast does not require progress in robotics, which could further reduce costs. I’ll focus on aerospace manufacturing, which comprises most of what the attacker needs to build: new missiles, surveillance constellations, and missile defense systems.

Can AI automate most intellectual labor? Aerospace projects of the type I’m describing here are highly interdisciplinary, and even simple systems demand expertise in electrical, computer, and mechanical engineering. More-advanced projects may require fundamental research in applied physics or math. While AI assistance for software development is a relatively mature use case, AI abilities in the other fields are much more nascent.

Yet there is good reason to believe that most intellectual fields will follow the same trends. AI abilities have increased at roughly the same rate across various domains, as shown in the chart below.

Additionally, we’re already beginning to see AI labs show interest in automating science and engineering fields: Anthropic is training on electrical engineering, and the company showcased 3D modeling performance in its Claude Fable 5 launch post. OpenAI has made discoveries in mathematics and physics. On the biology front, Google DeepMind’s AlphaFold has solved protein folding, enabling advances in drug discovery.

Extrapolating from such innovations, one engineer could do the work of many: outsourcing most of it to agents and handling only what agents can’t yet do, like running experiments or meeting stakeholders in person.

How much can automated intellectual labor lower military R&D costs? Although aerospace manufacturing seems like a labor-heavy job, it’s a remarkably white-collar profession, with high exposure to AI automation. In a 2021 article, Princeton University computer scientist Edward Felten and co-authors calculated that aerospace manufacturing has an AI Industry Exposure Score of 0.519, in the 70th percentile of all industries (around the same level as real estate). If AI automates the intellectual labor of aerospace manufacturing, this would substantially lower the cost of the final product.

To make things more concrete, consider SEC filings showing the financials of two public launch services companies, Rocket Lab and Firefly Aerospace.

R&D (which accounts for 28%–48% of spend at the two companies) is mostly engineering compensation and analysis, with the remainder going to propellant, test facilities, and hardware. It’s mostly cognitive labor, and therefore a potential target of future AI automation. SG&A expenses (21%–22% of spend) describe the cost of finance, legal, contracts, compliance, and business development. These disciplines rely almost completely on cognitive labor. Cost of revenues (31%–51%) is the most mixed category: it blends physical labor (technicians doing welding, assembly, and launch operations) with white-collar labor (manufacturing, quality, and test engineering), and purchased materials and components.

The costs of aerospace components are themselves quite compressible. For example, star trackers (devices used by satellites to measure their own orientation) can cost over $100,000. These devices’ physical components—a camera sensor, a housing, and some processing hardware— may cost as little as a few thousand dollars altogether. The more significant cost comes from qualification testing and R&D (because of low manufacturing volume, such costs can’t effectively be amortized). So, by reducing the cost of R&D, automating intellectual labor can drive down component costs.

I expect the cost savings to be even greater for defense aerospace products, which usually cost more than comparable commercial products. This higher cost partly reflects the increased documentation and security practices required for highly regulated uses. Such regulatory compliance practices rest almost entirely on cognitive labor that can be automated. Another driver of these product’s higher costs is costlier labor: defense contractors must have security clearances, limiting the worker pool and raising wages. An AI, on the other hand, must be cleared only once; it can then be scaled indefinitely.

What Could We Build?

Recall the six requirements that have kept a counterforce strike out of reach. Each one has held not because it was physically impossible, but because meeting it at scale was prohibitively expensive. Once cheap intellectual labor collapses those costs, an attacker with an AI lead could pursue several military megaprojects at once, each one solving a different requirement needed to launch a successful counterforce strike. This section will focus on the three requirements I expect to become significantly easier to meet.

Locating and destroying nuclear submarines. Research has shown that, in certain conditions, moving submarines leave a surface wake detectable by Synthetic Aperture Radar (SAR), a space-based radar system. A large constellation of SAR satellites would permit near-continuous surveillance of the entire world, regardless of time of day or local weather conditions. The US is already pursuing a SAR satellite constellation, and will launch its first satellite in 2028. The US could supplement this effort with a space-based constellation of Light Detection and Ranging (LiDAR) satellites. Each satellite would measure depth via pulses of intense light, detecting even stationary submarines (although that would require clear weather). Certain frequencies of LiDAR can detect submarines within 200 meters of the surface, within typical nuclear submarine operating depths. A dense constellation of LiDAR and SAR satellites could sweep the oceans, and expose submarines at scale.

However, while both satellite methods can reveal submarine positions temporarily, the only tool to keep track of them consistently would be underwater drones (often called Unmanned Underwater Vehicles, or UUVs). UUVs are currently limited by the difficulty of autonomous operation, but AI R&D would likely significantly improve this. The US has been funding research in this direction.

Defending against surviving missiles. A system with tens of thousands of interceptors prelaunched in space (similar to the 1980s Brilliant Pebbles concept, originally abandoned due to cost issues) could counter any missiles that are missed by the first strike. The US is already pursuing this, too, with its Golden Dome system.

Overcoming the nuclear taboo. The missiles of the Cold War had poor precision, so warheads with an explosive yield of hundreds of kilotons were common (designed to guarantee a silo kill). But, as mentioned, such heavy warheads would result in enough fallout to guarantee millions of deaths. In 2017, Keir A. Leiber of Georgetown and Darryl G. Press of Dartmouth found that modern missiles have much higher precision than those from the Cold War, and future improvements could reduce the average targeting error to mere meters. With such high precision, very low-yield weapons could be used, with little to no fallout. A counterforce strike could be accomplished with an estimated death toll of around tens of thousands rather than millions, well within the range of wars nations are willing to start.

Other aspects of the retaliator’s deterrent are vulnerable to AI R&D too. The retaliators’ mobile missile launchers can be located with the exact same SAR constellation that we discussed for submarine detection, leaving them vulnerable to a first strike. Launch on warning systems are also vulnerable. While it’s unlikely they could be hacked outright, Anthropic’s Mythos demonstrated AI driven vulnerability discovery that could be used to confuse, delay, or reduce confidence. Additionally, the defender’s early warning response time can be shortened significantly with depressed trajectory submarine-launched missiles, which could cover 2000 km in only 10 minutes. The combination of high-speed and precise missiles also works to efficiently counter fixed silos.

The Defender’s Options

If a lagging nation realizes that a rival is on course to achieve nuclear primacy—the ability to execute a counterforce strike without retaliation—it still has a few options.

The first, and most straightforward option is to expand the arsenal. It can increase the number of silos, build more nuclear submarines and mobile missile launchers, and raise its level of alert. This strategy would work in the short term, since the attacker would be forced to scale up its own forces until it was certain it could neutralize all of the new forces. The problem is the cost-exchange ratio. If the attacker has an AI advantage, the retaliator could end up paying a larger price for each new silo than the attacker pays to build the missiles or interceptors that could defeat it.

The second option is to target the root cause: the AI gap between the attacker and retaliator. Potential avenues in this direction can range from relatively diplomatic to highly escalatory. Options include disrupting the attacker’s supply chain, launching data poisoning attacks, or sabotaging its AI training runs. However, the most extreme actions—direct kinetic attacks on datacenters or researchers—would be likely to start wars. But even these interventions will only be effective if applied early. Once the lead is large enough, sufficiently smart AIs will already be trained. Thus, an AI-lagging defender must be alert enough to act before the AI capability gap becomes overwhelming.

Finally, the attacker and retaliator could negotiate a treaty. A bilateral arms-control regime could in principle cap satellite constellations, ballistic missile interceptors, or AI compute used for military R&D. There is precedent here, especially in the nuclear domain. However, the main challenge will be aligning incentives. The leading nation has no incentive to join a treaty where only the lagging party stands to gain. Traditional arms-control treaties have only worked where there were symmetric costs on both sides.

AI May Disrupt Nuclear Deterrence

In conclusion, the historical robustness of nuclear deterrence has rested on a cost-exchange ratio that favors the retaliator, but AI-assisted R&D can invert the ratio. This might enable a single superpower leading in AI to achieve nuclear primacy. The AI superpower would then wield enormous leverage, as it could credibly threaten to win just about any war. Such negotiating leverage could reshape the global balance of power, even if nuclear weapons are never used.

Even before military infrastructure megaprojects are complete, they will affect policy. If major powers come to believe that AI R&D may make their nuclear deterrents less effective, they will have incentives to expand arsenals, shorten decision timelines, rely more heavily on launch on warning, contest one another’s space architectures, and target the AI and semiconductor bases that underpin their adversaries. This new equilibrium would increase military spending, shorten decision times, and raise the risk of war.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Govind “Vinny” Pimpale is a research fellow at the Foundation for American Innovation, where he focuses on AI policy. Before joining FAI, he worked at a startup developing reinforcement learning environments, and prior to that, as an AI evaluations researcher. He holds a BS in Computer Science and Engineering from UCLA.

On June 12, the Trump administration issued an order requiring Anthropic to suspend access to its two most advanced AI models—Fable 5 and Mythos 5—for non-American nationals, just days after their public release. The decision caused immediate backlash across Europe. Politicians from France’s Gabriel Attal and Jordan Bardella to the Netherlands’ Geert Wilders, alongside European industry and civil society, amplified calls for AI sovereignty. Attal, the presidential candidate for Macron’s Renaissance party, even likened the shutdown of Anthropic’s models to Iran’s blockade of the Strait of Hormuz.

While such reactions are understandable, they underscore what we already know: Europe is behind on AI, overly dependent on the US, and vulnerable to unilateral decisions. Perhaps the renewed sovereignty rhetoric can help build the political capital needed to fix the continent’s AI positioning, but what Europe needs more urgently is a clear-eyed account of what to actually do. In the wake of the Anthropic episode, this piece separates what has genuinely changed from what is being overstated, while identifying where European policymakers should focus their attention.

Europe Should Have Planned for This

In recent years, AI has become increasingly relevant to national security. Europe could have anticipated that the US government would restrict access to US AI models at some point, but has not acted quickly enough to secure its position.

Export controls on US technology have affected Europe before. The use of export controls on American technology for national security reasons is nothing new and is not unique to the current US government. The Biden administration’s AI Diffusion Rule did the same for advanced chips, treating AI hardware as a national security instrument and using export controls to manage its global distribution. Europe was affected then, too, with Tier 1 countries in western Europe split from Tier 2 countries across much of eastern Europe. What the new controls do, less selectively than the Diffusion Rule, is extend export controls from AI hardware to AI models themselves.

The US implemented the new control suddenly and with opaque reasoning. While export controls are not unprecedented, the Trump administration’s recent directive differed from Biden’s Diffusion Rule. This decision happened more quickly, and was driven by harder-to-interpret motives, particularly in light of the recent clash between Anthropic and the Pentagon. It is plausible that the US government wanted to block Fable’s deployment for everyone (Americans included) and used export controls as a tool to do so.

Europe has been moving too slowly to build sovereign AI or secure access to frontier American AI. Europe could have planned more effectively for this moment by treating AI as a sovereign imperative and developing the resources and infrastructure needed to better serve this goal. Instead, the scaling down of EU plans for AI gigafactories and the successive delays to its Cloud and AI Development Act point to the complexity of moving at the pace of developments across the Atlantic. A separate but related issue is the need to access American frontier AI models for, among other things, hardening European infrastructure against cyberattacks. European leaders could have sought to negotiate guaranteed early access to highly capable systems. Yet the EU has lagged here as well, obtaining access to Anthropic’s Mythos model about two months after the company first started sharing it with a small group of American organizations to bolster cybersecurity.

Subscribe now

The Restrictions’ Short-Term Impact on Europe May Be Limited

Europe is not, for now, significantly worse off without access to Fable 5 and Mythos 5. This is for two reasons.

Access to Fable 5 and Mythos 5 will probably be restored. The US government and Anthropic will likely come to some sort of agreement to enable the continued rollout of these models. For one thing, the export restriction also applies to foreign nationals working inside American labs—employees crucial to model development. Maintaining the controls would be akin to telling Anthropic to stop developing future AI models.

Other models are good enough for most current applications of AI. Other than some cyberdefense uses, most tasks across the European economy do not currently require the most capable Anthropic models. Alternative models can perform many of the tasks that users would have assigned to Fable, at a similar level of competence. OpenAI’s GPT 5.5, whose capabilities are comparable to those of Fable, remains free of export controls. Open-source alternatives, only months behind, also remain available to European markets. Choosing an AI model is ultimately a practical calculation: the right capability, at the right price for the task at hand. That is why AI adopters often prefer cheaper Chinese models over frontier American ones: they choose based on the job they need to complete, rather than simply selecting the models that sit on top of a capability leaderboard.

Hence, the economic drag Europe will face by losing access to Fable is likely to be modest in the short run. It will compound only if the restriction on European access to American frontier models continues and the capability gap between American models and alternatives significantly widens, neither of which seem likely in the near term.

In the long term, Europe needs frontier AI for cybersecurity and economic competitiveness. Even if the model restrictions’ negative impact can be absorbed for now, the European market will suffer if it cannot access leading models like Mythos and Fable. Aura Salla, a Member of the European Parliament, has argued that “Europe is better off without these models,” because they pose significant cybersecurity risks, with insufficient safeguards. But access to frontier models is an absolute prerequisite for defensive security, allowing domestic companies and governments to identify and patch vulnerabilities before attackers find and exploit them. In the long term, frontier access is also an economic necessity to prevent Europe’s industrial base from being relegated to secondary tiers of productivity. Being locked out of the most capable AI models is categorically different from Europe choosing how to adopt AI on its own terms.

In the Long Term, These Restrictions May Provide Opportunities for Europe

Counterintuitively, the US government’s recent move may turn out to be good for Europe. While a gradual loss of access to American frontier AI might have been anticipated, that eventuality may never have felt urgent enough to prompt serious action. This sudden, unexpected loss of access will be a sobering warning.

The US government’s directive underlined its power to withdraw access. Export-controlling Anthropic’s leading models revealed who holds the kill switch—and how easy it is to press. Even if the controls are overturned, the intent and execution is visible to every government in the world. The use of export controls to restrict global access to AI models signals the distinct power of the American executive branch in shaping the trajectory of AI.

The more dependent Europe is on American AI, the more damaging restrictions could be. Fortunately, for now, Europe enjoys a narrow window of insulation: unlike cloud services, leading American AI models have not yet been deeply woven into European public administration or critical infrastructure. Access to American AI, like access to the nuclear umbrella, may come at a cost to those that accept and depend on it.

The episode is an opportunity to drive political momentum toward middle power coordination on AI. Even if the US directive brings only limited short-term consequences, it illustrates how vulnerable Europe could be in the future if it does not start working to secure frontier AI access now. Countries within the EU, including France, Germany, and the Netherlands, could do more to engage other middle powers like the UK, India, South Korea, Japan, and Canada. They can coordinate on how they wish to govern this technology, secure access to the frontier, and shape how AI affects society. Even if these countries play different roles in the AI stack (and even though existing coordination channels between these countries are currently scant), they share a common interest in preventing any one government from dictating access and governance unilaterally.

As part of a coordinated agenda on AI governance, middle powers could align on their evaluations regimes and incident monitoring practices, while slowly building up shared leverage through collective procurement standards and investment in each other’s ecosystems. A recent example is Aleph Alpha and Cohere’s merger. More ambitious moves could draw on the model of the Eurofighter Typhoon program: pooling procurement across multiple governments to build collective infrastructure or capabilities that no single country could generate alone.

Europe must also navigate public-private partnerships between Washington and American frontier labs. The Mythos episode marks a turning point in government-industry relationships on AI. Despite having been working with government officials and American industry to secure the world’s most critical software, Anthropic had to take its leading models offline immediately at Washington’s request. The age of governments ignoring capable model releases is likely over. The question is whether oversight will be principled and based on scientific thresholds being crossed, or reactive and politically driven. For now, absent federal laws on AI, the latter seems more likely.

The right response everywhere is to invest in building a more mature evaluation ecosystem, enforce actual regulations (like the EU AI Act), and establish regular and transparent predeployment engagement between governments and AI developers—so that policymakers are not scrambling to assess model capabilities in the days after a release. Beyond nurturing positive oversight mechanisms, European governments would also do well to consider how to respond to a reality where consequential AI decisions are made in closed-door conversations between the US government and US AI companies.

How Does Europe Build Leverage When It Needs Results Now?

AI is progressing rapidly, and Europe must act quickly to improve its position. Fortunately, it has a number of tools at its disposal to maintain access to frontier AI models.

Leading AI companies need compute capacity, and Europe could provide it. First, Europe could seek to build compute capacity quickly, along with the energy infrastructure necessary to power it. The aim would be to use this compute not exclusively for European AI development but as infrastructure that reduces dependence and creates negotiating leverage. The constraint on frontier AI development and deployment is increasingly compute, and evidence of its scarcity keeps accumulating. In March 2026, Anthropic tightened peak-hour session limits for paying users due to capacity constraints. By May, it had agreed to spend $1.25 billion per month to rent capacity from xAI’s Colossus cluster through 2029. When a leading AI lab is forced to buy infrastructure from a direct competitor just to keep its basic tiers online, it is a clear signal that compute remains a constraint to AI development.

An AI lab that needs European data centers, European talent, and European revenue has reasons to treat Europe as a partner rather than a market it can afford to work around. If Europe becomes a major host of leading models, this makes restrictions less likely due to the costs to American companies of losing access to the European market. Not so long ago, Nvidia was allowed to sell chips to China because of its importance as a market. To build such leverage in practice, Europe needs to urgently address the barriers to buildout: high energy costs, slow planning processes, and fragmented capital markets.

Middle powers could cooperate more deeply to demonstrate their importance to AI development. Europe should work with other middle powers to mobilize a coordinated response. The semiconductor supply chain runs through the Netherlands, Japan, South Korea, and Taiwan. AI evaluation and testing capacity is concentrated in the UK. Many of the most commercially significant AI applications—from Lovable in Sweden to leading adopters across Canada and India—are built on top of American frontier models, generating the revenue and usage data those labs depend on. Export-controlling American models while depending on allied components, talent, testing infrastructure, revenue, and data should be a strategy with a limited shelf life. Middle powers need to work more closely to demonstrate that they are as necessary to producing AI as the companies currently building it.

Europe must both invest in AI sovereignty and build relationships with US AI companies. Finally, Europe should amplify efforts to build indigenous frontier capabilities and chips, even if the near-term prospects appear improbable. An ambitious European AI effort would require spending levels that dwarf anything Europe can credibly mobilize through subsidies alone. Meta will invest $125 billion in capital expenditures this year, more than Germany’s entire defense budget, and its AI models still fail to match capabilities from OpenAI and Anthropic. But, difficulty aside, meaningful European AI development will become a necessity over the long term. The question is how Europe can build sovereign AI capabilities while strategically developing partnerships with value-aligned US companies, such as Anthropic. Doing neither or only one has too many downsides; doing both buys optionality.

Export Controls Are a Wake-Up Call for Europe

Arguably, the export controls on Fable 5 and Mythos 5 do not fundamentally change Europe’s position on AI. Europe was dependent on American AI models and chips before last Friday and remains so today. The immediate economic consequences are likely manageable, particularly if the restrictions prove short-lived. But Europe’s response must nonetheless move beyond the familiar debates on sovereignty. It should be concrete: expand compute capacity, coordinate closely with other middle powers, invest in domestic capabilities, and deepen partnerships with firms that need European markets as much as Europe needs them. The overall impact of the recent order could in fact be positive for Europe—but only if it spurs the continent into acting on AI in ways that it already needed to.

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Afek Shamir is an Analyst at RAND Europe, working with the Center on AI, Security, and Technology and the Frontiers of Technology hub. His research focuses on Europe’s role in frontier AI development and governance, with particular interest in the continent’s geopolitical positioning and leverage vis-a-vis other leading AI powers. Prior to RAND, Afek worked at a Brussels-based think tank on the EU AI Act’s governance of general-purpose AI as a Talos fellow and interned at the Tony Blair Institute. Afek holds an M.Sc. in European and International Public Policy from the London School of Economics.

The debate about AI’s future economic impacts often settles into two camps predicting incompatible futures. One camp insists that AI is a normal technology: simply the next in a long line of economic transformations, each increasing productivity while gradually reallocating labor. The other camp warns that AI will become a great displacer: that automation will hollow out the working class within a decade and eventually disempower large swaths of human workers.

Each side often treats the other’s predictions as unserious, and, consequently, policy debates often split along the same tired fault lines: whether we need reskilling or universal basic income, whether we should strengthen safety nets or structurally redesign our economy. The two camps’ forecasts diverge so sharply that it can be hard to see that they do not have to be mutually exclusive.

Rather, a more useful framing treats these predictions as describing different stages of the same overarching transition rather than as competing accounts of the same moment. From a macro perspective, both narratives will play out roughly sequentially, though those phases may overlap substantially across sectors and timelines.

In the short term, it seems inevitable that AI will look like an accelerated version of past automation waves: significant productivity gains after a period of integration, job displacement in specific occupations, and a familiar churn of workers cycling into new roles.

In the long term, it is hard to conceive of a future in which transformative AI systems do not lead to a massive restructuring of the economy and a reconsideration of the role of human labor. A world in which machine intelligence can perform most economically valuable cognitive (and, increasingly, physical) labor, at a fraction of human cost, must eventually lead to a completely new kind of economic system.

Our responsibility during this period is to prepare and to guide our economy deliberately through these sequential and overlapping transitions. To do so, we must develop thoughtful roadmaps that account for both near-term and long-term impacts, and that can adapt effectively to support national governments in managing these changes.

In the rest of this article, we lay out such a roadmap, describing each phase of the economic transition and outlining some of the most commonly discussed policy solutions at each stage.

Near Term: Managing Economic Shocks

In the near term, the most pressing economic concerns are AI economic shocks and the labor displacement of certain groups, such as early-career employees or workers in highly exposed occupations. Certainly, there will be other jobs to transition into. The only question is whether they will be accessible or desirable.

Initial displacement will be concentrated in certain domains. The impact of AI will be highly varied across sectors, with some being significantly more vulnerable than others. A recent report from Boston Consulting Group estimates that around 50% of American jobs will see restructuring or reshaping due to AI. For the average white-collar employee or college graduate, what their career will look like in five years is quite unclear.

Displacement could be sudden. In particular, markets are concerned about a potential rapid collapse of demand for historically well-paying occupations such as software engineers, financial analysts, and legal associates, which could trigger cascading effects. Research suggests that up to 90% of automation-related job losses occur during the first year of recessions. If an increasingly unequal economy encounters a sudden slowdown, labor displacement could be both sudden and concentrated. The initial shock would be further compounded by the second-order effects of reduced tax revenues, weakened consumer demand, and wage scarring (the long-term negative impact of unemployment on an individual’s wages).

One remedy is to modernize and scale active labor market policies. Among the most popular solutions to these near-term risks are policy interventions aiming to help people find and keep jobs. Wage insurance programs have shown promising empirical evidence to improve worker outcomes during transition periods. For instance, Germany’s Kurzarbeit helped it avoid rising unemployment during the 2008 financial crisis, making it the only G7 country to do so. Meanwhile, the US’s Reemployment Trade Adjustment Assistance program is estimated to have increased employment probability by 8% to 17%, and it largely pays for itself through higher tax revenue and reduced benefit outlays.

Reskilling programs have also been widely discussed among policymakers, though it is still unclear what industries workers should be retraining for. Other proposals include dynamically expanding unemployment benefits or job guarantee programs that could provide transitional public employment.

To strengthen these programs, governments must invest more significantly in labor market data, streamlined benefits systems, and payment infrastructure—the absence of which hampered COVID-era relief distribution globally. These investments, made in the near term, can also help to develop the infrastructural backbone for more ambitious medium- and long-term interventions.

Medium Term: Navigating Reorganization and Divergence

In the medium term, the transition to an economy dominated by AI will present both extraordinary opportunities and structural risks. As AI systems become increasingly capable, they will be able to complete ever more workstreams end-to-end, potentially driving broader job displacement than seen in the near term. A new class of superstar firms might emerge in winner-takes-all markets where scale—especially of compute and capital—could confer decisive advantages. The medium-term period could be defined by a delayed but rapidly accelerating impact on productivity and employment, an increasing divergence in AI adoption and growth between regions and countries, and growing pressure on fiscal systems.

Differences in AI adoption may drive divergent outcomes for countries. Since countries will adopt and develop AI unequally, the impacts on productivity are also likely to differ. This could contribute to increasing global inequality, especially when technological diffusion is limited (e.g., by export controls, protectionism, or regulatory barriers). The White House Council of Economic Advisers warns that countries lacking the ability to develop advanced AI face compounding disadvantages that could produce a second Great Divergence, paralleling the Industrial Revolution.

Widespread labor displacement could substantially impact tax revenue. If new economic growth is increasingly captured by a smaller proportion of AI-led corporations, tax systems built primarily around payroll taxation could face revenue shortfalls and a structural mismatch between where value is created and where it is taxed. Globally and in the US, labor revenue is typically taxed at a significantly higher rate (and more effectively) compared with how capital is taxed. A substantial shift of economic growth toward capital could therefore lead to multi-digit percentage declines in revenue.

Several common themes in today’s policy conversation could help to address the medium-term economic impacts of AI.

Policymakers will need to begin considering taxation reforms. Leading economists have proposed a shift toward consumption-based taxation if labor income declines in importance. By capturing spending rather than earning, higher consumption taxes would sidestep the question of labor versus capital income. A contrasting approach centers around progressive corporate taxation, which would impose higher marginal rates on the most profitable multinational enterprises through global coordination. Alternatively, token taxes could capture revenue streams directly from leading AI corporations. Any of these measures alone might be insufficient; an effective fiscal response would likely need to combine approaches to create a tax code with adequate resilience and fitness for future scenarios.

Countries could strengthen their economies by fostering AI-related industries. As economies grow more focused on AI, countries will need to consider active industrial policy, primarily to aid their economic competitiveness and stimulate job creation in increasingly important industries. Ramping up AI infrastructural investments is already a mainstream discussion in most countries, where we are seeing proposals to increase capital inflows, create special “AI Growth Zones,” and subsidize data center investments. Emerging ideas include publicly owned AI foundation models or tax breaks incentivizing employers to invest in worker retraining or human capital development.

Governments may need to actively protect vulnerable workers and industries. Labor subsidies targeting socially valuable sectors could preserve employment where human participation generates positive externalities, such as education or elderly care. For example, in 2021 South Korea’s Senior Employment Program provided work for roughly 840,000 people over age 60. Elsewhere, leading economists have proposed a series of policies to encourage “pro-worker AI”: deploying assistive AI that makes workers more productive, instead of outright replacing them. Strengthening collective bargaining rights may also play a key role in determining whether organized labor can secure meaningful leverage for workers.

By making such investments in the next decade, governments will determine whether they can protect workers in a medium-term future where new economic growth becomes increasingly dominated by capital. These measures could also lay the foundation for managing the largest economic impacts of AI over the long term.

Long Term: Restructuring Economies

Provided we avoid the more extreme risks of AI, a likely trajectory of the technology is that it eventually surpasses humans across an increasing proportion of economically valuable tasks. Machine intelligence faces fewer fundamental constraints than its biological counterpart. AI will continue to decrease in cost both for cognitive labor—which is already price-competitive with humans on many tasks—and eventually for manual labor, which will be constrained primarily by the marginal cost of robotics systems.

AI could automate a steadily increasing proportion of new economic growth. In the long run, there may be few persistent bottlenecks to automation as the economy restructures around powerful AI systems. Durable human advantages may persist primarily in domains requiring interpersonal connection or physical presence, as well as in contexts where people specifically prefer human involvement.

Automation could change the social contract. This transformation could challenge the foundational premise of modern economies: that hard work and talent are the primary route to income and economic security. If labor ceases to be a reliable path to capital accumulation, core aspects of the social contract may break down for a growing share of the population. Eventually, the policy challenge may shift to fundamentally redesigning the relationship between citizens and the economy itself.

Many ideas for meeting this challenge have been suggested, beyond the call for universal basic income.

Equity and capital may need to be predistributed. Many recent proposals have centered around fractional public ownership of AI equity, requiring that AI firms transfer equity to governments, which would own them on behalf of the public. Unlike universal basic income, which requires perpetual political will, redistributing capital would create durable property rights that compound over time. Leading economists such as David Autor and Neil Thompson argue that we should begin experimenting with universal basic capital (UBC) now, as, even in the most ambitious cases, it would take multiple decades for capital ownership to be broadly diffused.

Sovereign wealth and international coordination may help to distribute AI benefits. Sovereign wealth funds (SWFs) have emerged as a widely discussed institutional vehicle for public co-ownership, with promising examples in Norway and Alaska. By holding equity stakes in AI firms and infrastructure, governments could become better invested in the long-term success of AI and eventually distribute these gains to citizens via cash dividends or public services. Globally, an increasing divergence between countries that produce AI and countries that consume it may eventually lead to calls for stronger international institutions, multilateral tax coordination, or even dividend funds on behalf of all humans.

Governments may provide universal basic services. In the long run, governments may choose to expand the direct provision of essential services—including health care, child care, and education—so that they are fully decoupled from employment status. The UK’s National Health Service, Finland’s free university system, and Vienna’s social housing model demonstrate that universal basic services can be administratively feasible and politically durable.

With a combination of these policies, it is entirely plausible that in highly automated and productive futures, governments could ensure a basic level of economic security for all citizens. The open question is whether the political will to do so will exist.

Conclusion

The exact policy interventions will differ dramatically on a country-by-country basis. There is no single policy roadmap that will work everywhere; each government will need to design a strategy uniquely suited to its own citizens, culture, and institutional context.

Each stage of interventions can help create the infrastructure for the next. In many cases, the policy proposals described above help to lay the groundwork for navigating later stages of the economic transition, as well as having immediate benefits during the stage at which they are implemented. Building social safety nets today may enable greater bargaining power for labor later. Strengthening taxation mechanisms eventually supports funding for broader public service provisioning. Effective economic policies compound: they succeed as deeply interwoven networks over decades of investment.

Nations will need to develop their own economic preparedness plans. Governments should develop self-assessments and policy strategies tailored to their specific labor market exposure to AI. By evaluating a wide range of potential scenarios, countries can test their preparedness for highly uncertain futures. Only with that foundation can they develop strategic policy roadmaps for the transition ahead.

Policymakers globally are just beginning to recognize that the intersection of AI and labor will be a defining theme of upcoming elections. Within a few years, this will likely become a core issue for political candidates around the world. Yet governments are not remotely prepared to offer coherent responses on the scale these challenges will require. If we can support our policymakers with better foresight and more coherent roadmaps to economic success, we may be able to guide this upcoming transition toward prosperity and widely shared financial security.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Deric Cheng is the Director of Research for Windfall Trust, a non-profit focused on ensuring that the economic benefits of advanced AI are shared by everyone. He is also the lead for AGI Social Contract, a consortium of experts proposing strategies to design a new social contract for a post-AGI society.

Jacob Schaal an economist researching AI’s labor market impacts. He is a researcher at Kings College London, and co-edits the AI Economics Brief at Windfall Trust. He holds an MSc in Economics from the London School of Economics.

“Bloodthirsty AI models more willing to start nuclear war than human counterparts.”

It seems almost inevitable that any media headline about AI will be hyperbolic. Yet this statement, taken from a February 2026 New York Post headline, was accurate. The alarming claim stems from a widely publicized study by King’s College London, which found that, in simulations of international crises, LLMs reached for the nuclear trigger 95% of the time.

This academic study drew mainstream-media attention because it touched upon a cultural narrative that has long combined the concept of AI with nuclear weapons. Arguably, the first movie to bring the two together was 1957’s “Invisible Boy,” featuring Robby the Robot, who would later become famous (and less bloodthirsty) in the 1960s TV series “Lost in Space.” The trope has since been repeated across franchises ranging from “The Terminator” to “Mission Impossible.”

Yet the AI-nuclear fear is not confined to the media and movie theaters. The King’s College study is only one of scores of similar academic and think-tank research projects on AI’s proclivities for nuclear war, which have been backed by millions of dollars in research grants. Among the entities that have funded such work are the National Nuclear Security Administration (NNSA), the Department of Energy, the Department of Defense, Anthropic, the MacArthur Foundation, the Carnegie Corporation of New York, the Future of Life Institute, Open Philanthropy, RAND, and the Smith Richardson Foundation. It is also an essential element in the larger field of study on the existential risks of AI, funded to the tune of multiple billions of dollars at many of the world’s leading universities, including Oxford, Cambridge, Stanford, and the University of California, Berkeley.

Beyond academia, the discussion of AI-nuclear risks has also entered the halls of government in settings that range from the UN to multiple US-China superpower summits to the US Congress. It has even become part of the reasoning for why states like California are seeking limits on frontier AI. Indeed, the fear of “the incorporation of AI into nuclear decision-making” has reached such a height that the “Bulletin of the Atomic Scientists” earlier this year moved its famous “Doomsday Clock” to 85 seconds to midnight, the closest it has ever been. (For comparison, the clock was at 12 minutes in the aftermath of the Cuban Missile Crisis.)

Despite all this, I am excited to be the bearer of good news: AI is not going to start a nuclear war anytime soon. I do, however, also have bad news: AI is making it more likely that humans will start a nuclear war. And, if we want to avoid that outcome, we should focus on mitigating real risks, resisting incentives that steer us toward the tropes.

The following report will explain why no government will likely delegate nuclear launch to machines either now or in the future, and identify three mechanisms—arms racing, miscalculation, and machine speed—by which AI could already be amplifying the risk of humans deciding to go to war.

Why AI Will Not Decide Nuclear Wars

Studies such as the one from King’s College consider hypothetical scenarios in which an LLM determines whether a nation should proliferate nuclear weapons or even when to fire a full nuclear strike. Yet there are multiple good reasons why humans will not, in reality, hand machines this responsibility.

Adversarial robustness and a dearth of training data present technological barriers. First, to be trusted with nuclear decisions, AI systems would need to not only be more capable than humans but also extremely adversarially robust—meaning that geopolitical opponents could not influence such systems by manipulating input data. While capabilities have come a long way, adversarial robustness has historically proven very challenging. Another glaring issue is the availability of training data for fighting nuclear wars: there is none.

There are laws and agreements prohibiting fully autonomous nuclear decisions. Alongside the technological barriers are domestic laws and international agreements. Section 1638 of the US FY 2025 National Defense Authorization Act calls for keeping a human in the loop for nuclear decisions. Similarly, Chinese President Xi Jinping and then–US President Joe Biden agreed in late 2024 that AI should never be granted the authority to initiate a nuclear launch.

AI technology may advance and laws could certainly change or be ignored. But one thing will not: the use of nuclear weapons will always come down to decisions of politics and war. This is critical. It explains why AI systems are not presently in the position they occupy in the studies and movies and why they will not occupy that position in the future.

Take the area of politics. In the 2025 film “A House of Dynamite,” a fictional US president facing a nuclear crisis laments that he and his team spent more time on a Supreme Court justice nomination than on whether to launch a nuclear strike—the most important decision of not just his life but maybe human history.

What the various scenarios and studies require is that a human leader either consults an AI for advice, when weighing a potential nuclear strike, or has already turned over this decision to a machine at some point beforehand. Either would be the most unlikely of political outcomes, in any form of government outside the imaginings of Silicon Valley techno-state.

Political leaders would not give up the most important decision of their lives to an AI. Political leaders would neither surrender nuclear decision-making to a machine nor blindly follow its advice contrary to their own judgment. To think that elected leaders would defer to an AI even on whom to nominate to the Supreme Court, let alone on a nuclear launch, is to assume that such leaders lack confidence in themselves and their judgements—something which political leaders are not in short supply of. This is why it was so easy for Biden and Xi to agree not to let AI decide nuclear launch; it is something they would never have considered anyway.

Any suggestion to put AIs in charge of nuclear decisions would spark public outcry. Moreover, even to contemplate creating the policy and technical mechanism for changing the multiple-billion-dollar, three-generations-old nuclear command-and-control architecture would create career-ending outcomes for democratically elected leaders. Only 3% of Americans believe that we should trust information from AI even “almost all of the time.” A leader proposing to trust it with nuclear war could expect to be rapidly relieved of any power to implement that proposal.

Authoritarian leaders are even less inclined to hand over consequential decisions to anyone or anything other than themselves. This is even codified in some nations’ command structures. Under China’s “Chairman Responsibility System,” the operational authority to command or launch nuclear weapons rests solely with the chairman of the Central Military Commission, who just happens to be the general secretary of the Chinese Communist Party.

Militaries are rapidly adopting AI, but not for nuclear decision-making. Of course, militaries are spending billions of dollars on AI, but not on integrating it into nuclear decisions. As a study on the “nexus of nuclear weapons and AI” summed, “AI is unlikely to have material impact on nuclear command and control, which for several decades have synthesized automation but not autonomy.”

Rather, like institutions across the economy and society, militaries are turning to AI to save on money, time, or head count, and/or to get better at the tasks they are not good at. Nuclear warfighting is not an area where militaries feel those pressures.

To begin, the vast majority of militaries’ AI applications are off the battlefield in support roles, usually with civilian parallels, ranging from military medicine to military logistics. Such support roles make up over 90% of military jobs.

Militaries need AI to assist in dynamic, non-nuclear warfare. Applied to warfighting, the US military has been focused on closing the “OODA loop” (Observe, Orient, Decide, Act) in non-nuclear war, through enhancing sensors, expanding analytical capabilities, and accelerating the application of this process to a target to strike or defend against.

Consider the 2026 Iran-US war. The war began in an era with AI, and may have been partly about nuclear weapons. However, neither the decision to go to war nor the use of AI in the war have been linked to any of the worries that have consumed so many grant proposals and reports. AI was used by all parties, but in nothing like the way it is used in the scenarios. US and Israeli forces tracked hundreds of thousands of moving parts, analyzing both their own forces and their potential targets, and then parsed them out for thousands of strikes. In the first four weeks of conflict, US forces struck more than 10,000 targets, while Israel hit thousands more.

In turn, Iran used AI to aid in its tracking and targeting of everything from tankers to US helicopters, firing off thousands of drones and missiles. The conflict then moved into a cat-and-mouse game in which Iranians sought to fire hidden missiles or drones before loitering US drones found and destroyed them. Each side would then try to analyze whether it had destroyed its target or needed to repeat the attempt. One military analysis described the machine-speed cycle as a “90-second war” taking place over multiple months.

Nuclear weapons, by contrast, are both physically designed and organized in military doctrine to be used against large and usually pre-decided targets. They are not used in repeated short-loop cycles over campaigns of weeks or months. To put it another way, the military needs AI to help it find a needle in a moving haystack, pull that one needle out, attempt to snap it, and then determine whether it was snapped. The military does not need AI to find the haystack sitting in a field, set it on fire, and then know whether or not it burned down.

These non-nuclear visions of AI are shared by other nuclear powers, including China. Over the last decade, the People’s Liberation Army has pursued an “intelligentization” program that integrates AI as a decision aid for rapid attacks on the enemy’s “kill chain”; AI-coordinated swarming drones to overwhelm defenses; and cognitive warfare that uses AI to target human minds through information and cyberattacks. The PLA Information Support Force is building a “network information system” that uses AI, cloud computing, and big-data techniques to fuse data from operational units and create “dynamic kill networks” across air, land, sea, space, and cyberspace domains. AI is not, however, in charge of nuclear weapons. Indeed, in a study of over 9,000 AI-related requests for proposals published by the PLA, establishing its AI “wish list,” projects using AI for nuclear decisions did not feature once.

On the contrary, running through China’s military approach to AI is a goal to steer military operations from Beijing. Far from delegating all war choices to machines, Chinese leaders very much want machines that follow the orders of humans—specifically, that same decision-maker in charge of nuclear weapons, the general secretary of the Chinese Communist Party’s Central Committee.

AI Arms Races: Spend More, Feel Less Secure

If we truly want to support global peace and security, we should not focus on the cinematic threat narratives in which AI controls nuclear decision-making. Instead, the more realistic concerns arise not from the mix of AI and nuclear weapons but from interactions between AI and humans. The first of these concerns is that the promises of AI are now fueling an arms race more intense than previous arms races.

The current AI arms race is heightening feelings of insecurity. A 2026 report by the US Center for Strategic and International Studies summed up the consensus among policymakers and researchers this way: “Conventional wisdom holds that an AI arms race will define the twenty-first century and could be decided as early as 2030.” This viewpoint has even been codified into the highest levels of state doctrine. The second Trump administration’s National Security Strategy explicitly proclaims that AI “will decide the future,” echoing Russian President Vladimir Putin’s 2017 statement that whoever leads in this field “will be the ruler of the world.” In China, too, achieving global leadership in AI is a non-negotiable national priority; the country’s most recent five-year plan pledges to use “extraordinary measures” to realize that goal.

What is playing out is a classic security paradox: as nations accelerate their capital investment to secure a technological edge, they raise the stakes and concerns for their opponents, ultimately feeling more vulnerable than when the race began. In short, the more you arms-race, the less secure you feel. But, while arms races are nothing new, AI differs from previous military technologies in ways that introduce three additional layers of insecurity.

AI is considered a winner-takes-all technology, intensifying the arms race. First, AI may confer a decisive advantage at a smaller capabilities lead than other technologies. Historically, a nation could trail an adversary technologically and even quantitatively but still stay in the race. For instance, at the turn of the 20th century, the race for dreadnought battleships exacerbated tensions between Imperial Germany and Britain and became a contributing factor to World War I. Notably, though, the German navy never built as many dreadnoughts as the British. (Between 1908 and 1912, Britain launched 29 capital ships, while Germany launched 17: just under 59% of its opponent’s number.) Yet this disparity didn’t keep Germany from fighting the Royal Navy to a draw at the 1916 Battle of Jutland and maintaining the threat of a “fleet in being” for the rest of the war. So too in the nuclear age, China has maintained deterrence against the US with only 16% of the warhead count—approximately 600 warheads against the US arsenal of 3,700.

With AI, however, policymakers appear to perceive domination in binary terms: falling behind is equated with a total loss of strategic agency. Leaders would not tolerate having only 16% or even 59% of their opponent’s technological capability. On the flip side, a leader who believed their military AI capabilities were 85% better than those of their foe might feel invincible.

AI introduces a fear of falling behind permanently. Second, exacerbating insecurity even further, that perceived binary domination may prove real. At some point, an AI that darts ahead could conceivably become infinitely better than its competition, forever, potentially affording the leading nation a permanent decisive advantage. Regardless of whether this turns out to be true, it creates a fear of falling behind in a race where catching up feels impossible. Every technical milestone then becomes interpreted as a potential catastrophe. When the DeepSeek R1 model advanced beyond US expectations for Chinese LLMs, for instance, the discourse immediately framed it as a “Sputnik moment” for the United States.

Uncertainty about how best to use AI contributes to heightened concerns. Finally, AI’s versatility adds a layer of uncertainty about the smartest ways to use it. In past arms races, whether with ballistic missiles or battleships, the technology was largely uniform. The goal was simply to gain and deploy as many units as possible. AI, however, can diverge into radically different architectures and many more strategies. For instance, an article in “National Interest” warned that “America is running the wrong AI race,” by focusing on advancing frontier models, rather than on large-scale deployment of existing ones.

The overall result is that the AI age is creating a sense of extreme insecurity among many nations—an environment that is not conducive to peace.

The Cognitive Fog: Misperception and Miscalculation

A second concern about AI is its potential to add to the fog of war. Although AI is frequently marketed as a tool to provide clarity in complex analyses, it can also fuel misperception and miscalculation in multiple ways, potentially increasing the risk of humans deciding to go to war.

Militaries are using AI in sophisticated deception operations. First, as militaries integrate AI into their (conventional, not nuclear) battle plans, they are realizing they must also learn how to defeat opponents through new means of trickery. Recent PLA wargames have focused on how to “break intelligence,” preparing for battles in which AIs “work to distort each others’ reality.”

AI is being used for political deception. Second, AI is being used to dramatically scale up political disinformation campaigns. Conflicts in Ukraine, Gaza, and Iran have expanded to include what can be thought of as “LikeWar” battles to drive false information viral. Such campaigns involve automating information operations and creating high-fidelity deepfakes—tools that have already been used to successfully mislead heads of state, including those leading nuclear powers. US and Pakistani leaders have reacted to and pushed AI-generated imagery online. Studies on the present and future of deception operations suggest that this phenomenon will grow in scale and impact.

AI systems are vulnerable to errors, particularly in military contexts. The most dangerous and powerful kind of deception, however, is self-deception, which we can think of in both machine and human terms. AI undoubtedly brings incredible insights in various domains, often drawing on beyond-human analytical capabilities. Still, no matter how far the technology advances, these systems are plagued by issues that range from hallucination to algorithmic bias. Such problems, caused in part by training data that can never fully represent the real world, are especially salient in war. The civilian LLMs being brought into military systems are largely trained on the open internet—an objectively poor environment for high-stakes accuracy. Using military training data cannot solve the issue, since no two wars are the same. Datasets pulled from counterinsurgency operations in Iraq and Afghanistan, for instance, provide poor parallels for the conflicts of today and tomorrow.

A lack of understanding about AI could lead humans to make poor decisions. Yet, when it comes to miscalculation, the greater risk may lie with overly confident humans. History shows that the most dangerous phases of arms races are the earliest stages, when neither military nor political leaders yet fully understand the new weapons, and they make poor decisions based on erroneous assumptions. Before World War I, for example, the belief that new technologies like the railroad and fast-firing artillery gave a decisive advantage to the offense helped drive the quick march to war after the assassination of Archduke Ferdinand in Sarajevo. It turned out that these technologies in fact advantaged the defense, leading to four years of horrific stalemate in the trenches. A similar belief permeates discourse today—that AI rewards the side that strikes first (the offense) in every conventional war domain, from air strikes to cyberattacks.

It is more difficult to estimate opponents’ capabilities in AI than in other technologies. This risk is compounded by the challenges of understanding both one’s own capabilities and the enemy’s. Estimating power in the AI era is even more difficult than with traditional, kinetic weapons. Ships, tanks, planes, and even nuclear weapons can be counted, their physics understood, and their capabilities summed and compared. However, beyond estimating rivals’ data center capacity, understanding AI capabilities is far more difficult. Whether models can be accurately benchmarked is heavily contested. Even if accurate benchmarking were possible, how that would translate to battlefield performance would remain unknowable by humans or machines.

What we do know is that arms races traditionally incentivize exaggeration and fearmongering. Examples from the Cold War are the 1950s “bomber gap,” followed by the “missile gap”—Americans’ beliefs that the Soviet Union had achieved significant advantages in each technology. Both “gaps” turned out to be mythical, but they nonetheless contributed to the Cuban Missile Crisis. Similarly, both the George W. Bush administration and Iraq’s then-president, Saddam Hussein, issued claims about weapons of mass destruction that turned out to be nonexistent.

The outcome of direct interactions between opposing military AIs is unpredictable. Finally, the effects of interactions between AIs themselves create an informational void within military doctrine. In the past, adversaries generally understood one another’s concepts of fighting; with that awareness, they could deploy wargames and analyze recent conflicts to project outcomes. Because no nation will tip its hand regarding its true AI capabilities, and because military use of AI is relatively novel, the first time AI systems collide directly will likely be in a live, high-stakes environment where miscalculation is almost certain.

The Velocity of Catastrophe: Machine Speed

The third and final way in which AI is increasing the risk of war is through machine-speed operations. Yet, this is not about an AI making instantaneous decisions on nuclear strikes. Rather, it is about how AI is enabling a new generation of weapons that complicate humans’ nuclear decision-making.

AI is enabling weapons that compress the time window for humans to respond. The most notable examples of this phenomenon are boost-glide hypersonic weapons, including Russia’s Avangard, China’s DF 27, and America’s Dark Eagle. Despite their name, these delivery systems do not fly substantially faster than intercontinental ballistic missiles. What distinguishes them is that they use AI-enabled technologies, including adaptive control adjustments and cognitive and quantum inertial navigation systems, to make microsecond decisions and adjustments, while moving at thousands of miles per hour through denied airspace. By maneuvering around sensors and defenses, boost-glide weapons get much closer to their target before they are detected, allowing less time for humans to decide how to respond.

This “decision-time compression” represents the core of such weapons’ risk. Traditionally, Nuclear Command, Control, and Communications architectures provided a “decision window” of roughly 15–30 minutes to verify an incoming strike and weigh a response. To put this into context, the seemingly rushed time frame of “A House of Dynamite” spanned 18 minutes of deliberation, depicted by a 112-minute movie. A hypersonic weapon reduces the number of minutes for decision-making to single digits.

Shorter time windows often worsen human decision-making. The psychological reality is that humans make decisions poorly under stress and in short time frames, so anything that shrinks this window raises the likelihood of bad outcomes, such as a “Flash War.” There is a documented tendency for leaders to “lock in” on the first early concepts that enter the room—ideas that, in a more traditional crisis, might be aired out and debunked through hours of give and take. Historical experience confirms this; many of the most dangerous options considered by the US during the Cuban Missile Crisis were proposed during those frantic early days and then fortunately cast aside. AI-enabled weapons would not allow time for the same level of scrutiny.

Conclusions and Policy Recommendations

The notion that AI could start a nuclear war may be attention-grabbing. Yet research, grantmaking, and policy should be anchored in what is realistic rather than allowing the most dramatic narratives to steer the discourse disproportionately. The goal should be to understand and implement safeguards that tackle actual and likely risks, such as those posed by arms racing, misperceptions, and decision-making as described above.

Instead of pursuing purely symbolic measures to keep AI from nuclear weapons, we should prioritize reducing the incentives for and externalities of arms races, for example by finding ways to improve the defensive side of the equation. There may also be ways to reduce the likelihood of misperception, including through a concerted effort to enhance the education of political and military leaders about AI’s realities.

Finally, if we are to pursue effective arms control, our primary focus should not be on the speculative fear of AI launching a nuclear strike but, instead, on the real and growing number of physical platforms that shrink human decision-makers’ window for deliberation. For instance, prioritizing the regulation of hypersonic delivery systems—an existing technology that heightens the risk of nuclear catastrophe—is a more viable path toward strategic stability than chasing science fiction.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Peter Warren Singer is a Founder & Managing Partner at Useful Fiction LLC, a company specializing in strategic narrative, Strategist at New America, and a Professor of Practice at Arizona State University. A New York Times Bestselling author, described in the Wall Street Journal as “the premier futurist in the national-security environment” and “all-around smart guy” in the Washington Post, he has been named by the Smithsonian as one of the nation’s 100 leading innovators, by Defense News as one of the 100 most influential people in defense issues, by Foreign Policy to their Top 100 Global Thinkers List, and as an official “Mad Scientist” for the U.S. Army’s Training and Doctrine Command. No author, living or dead, has more books on the professional US military reading lists.

In 2017, Western media outlets warned that “Black Mirror is coming true in China.” The following year, Mike Pence claimed that “China’s rulers aim to implement an Orwellian system premised on controlling virtually every facet of human life—the so-called ‘Social Credit Score.’” So far, the CCP’s attempts at nationalized social scoring have remained fragmented and crude, largely due to difficulties in analyzing population-scale data. However, AI could soon lift that bottleneck, independently sifting through information and pulling out the most important details about every individual.

This unsettling prospect might renew fears about top-down social scoring by governments. However, an equally pressing concern is the potential for a bottom-up system, in which citizens choose to be surveilled and scored by AIs. As people integrate AIs into their lives to get more useful assistance with daily tasks, those AIs may soon be able to generate credible character assessments at the touch of a button. Early users who receive positive AI assessments may choose to share them with colleagues, businesses, bureaucrats, and so forth, in order to receive more favorable treatment. This dynamic would create an incentive for everyone else to follow suit.

This essay will explore why people will give AI assistants pervasive access to their lives and how this could soon translate into a form of social scoring. We’ll then map out how pressures to opt in will grow organically across every domain of life, creating a slippery slope toward self-imposed surveillance.

The Pressures Driving Self-Imposed Surveillance

The thought of sharing an AI judgment based on extensive personal data may sound too uncomfortable to believe that people would opt in. However, people already frequently give up their personal data to obtain benefits. In the US, over 21 million drivers voluntarily share driving data with insurers like Progressive and State Farm in exchange for discounts of up to 40%. Meanwhile, in China, voluntary disclosures have surged even as the country failed to implement a top-down, nationalized social credit system. In 2015, a private company called Ant Group launched Zhima Credit—an opt-in service that gives users social scores, and grants high-scoring users advantages from priority loan approval to dating site access. The platform claims to have more than 700 million authenticated users.

People are already sharing large amounts of personal data with AI for practical reasons. Some LLM power users are rushing to share their personal information with LLMs, connecting their agents to online accounts, medical records, and even bank information in hopes of obtaining more informed and wide-reaching assistance. Indeed, there are already wearable AI devices that can constantly record users’ lives, offering summaries of each day and making personalized plans for the next. From managing schedules to preserving an infallible, easily searchable memory of every conversation, AI agents are proving to be useful personal assistants in people’s busy lives. Adoption has already begun, and it is likely to expand.

Future AI assistants could provide attestations about their users. While people will initially share their personal data with AIs to get practical assistance, more capable future AIs could use this information for more than just helping with day-to-day tasks; given enough access, they could offer character references attesting that their users are reliable at work, committed as friends and partners, and honest in their financial and legal dealings. Once a user has granted their AI assistant wide-ranging access to their life, generating an assessment may be as simple as clicking a button.

After initial adoption, everyone else is under pressure to follow. Once early adopters disclose, others are likely to follow. This is due to an effect known as “unraveling,” which was described in two papers published by economists Sanford Grossman and Paul Milgrom in 1981. The logic is straightforward: those with the best track records have every reason to share them, since doing so distinguishes them from the crowd. Once the best disclose, everyone else looks worse by comparison, so the next-best tier discloses too. This cascade continues until nearly everyone has disclosed and silence itself becomes a red flag.

The transition to self-imposed surveillance could start small. While most people would balk at the end result of AI social scoring, it could nonetheless creep in gradually. The first use cases for AI attestation will be narrow and low-stakes: companies might let contractors attest to claims about previous projects, or dating apps might let users verify that their self-descriptions are accurate. Companies offering AI-based assessments might be widely disliked or little-used at first, but this would not necessarily block early adopters from opting in. FICO scores illustrate where this dynamic ultimately leads: no law requires you to have one, but opting out means losing access to housing, credit, and employment.

The Evolution of Self-Imposed Surveillance

Given the logic of unraveling, one might wonder why we don’t have complete self-surveillance and social scoring already. This can be explained by real-world frictions that make disclosure more difficult and less rewarding than it would be in theory. AI may soon remove these frictions, enabling unraveling not just in narrow domains such as driving and personal finance, but across every sphere of life.

We do not see full disclosure yet because there are frictions that block unraveling. Today, meaningful attestation often costs real effort, from assembling a job application to sharing references. This creates enough friction that a lack of full disclosure does not necessarily look suspicious.

Additionally, attestations are not always credible: landlords can overstate the quality of a rental, job applicants can embellish their qualifications, and there is often no practical way to check. This reduces the value of disclosure, since it does not reliably distinguish those with the best credentials from everyone else. Indeed, the domains where we already see unraveling are the narrow areas in which disclosure is both costless and credible. A FICO score captures financial behavior, and a telematics device captures driving behavior—information that is cheap to measure and difficult to fake.

Previous technological revolutions, such as the internet, sparked concerns about surveillance and social scoring. The rise of digital banking, health apps, online calendars, and social media means that large amounts of sensitive personal data is stored online. Yet we have not seen waves of disclosure across every area of life. This is because the internet does not in fact make disclosure costless and credible across all domains. On cost, analyzing vast sums of internet information to draw out valuable insights about every individual is still an intractable technological challenge. On credibility, the public internet is still a far cry from comprehensive, real-time surveillance of behavior. People can curate what they upload online, and this reduces trust that it is representative.

AI may make costless, credible disclosure dramatically easier. AI products like ChatGPT already store usage data that can speak to their users’ work competence, behavioral tendencies, and personal preferences. It could be very cheap and convenient to share such a profile (or a redacted summary generated by a trusted third party) with an employer or landlord.

Credible disclosure requires two things: comprehensive coverage of someone’s behavior, and the ability to draw accurate conclusions from that behavior. AI is making rapid progress on both fronts, particularly on coverage: heavy users already spend dozens of hours per week interacting with AI chatbots, and that coverage will only grow as companies roll out features to further personalize and integrate AI into daily life. Multimodal, always-on hardware like smart glasses and earbuds could grant AI assistants constant audio or visual access, and produce attestations far more credible than anything text-based interaction can support.

Societal pressures could make people accept comprehensive self-surveillance. Even as comprehensive surveillance becomes technologically feasible, people might feel uncomfortable about allowing their AI assistants to provide character assessments to others. Sharing driving data is one thing; sharing a judgment drawn from every detail of how one spends one’s day, from drinking habits to private political conversations, is quite another. Yet there are reasons to believe that resistance might yield surprisingly quickly.

Those who allow greater access will receive more credible attestations. Initially, users may try to game the system by granting AIs only selective access: interacting when they’re being productive and setting the AI aside when they’re not, or filtering access that may paint them in a bad light. For one thing, this would be a difficult strategy to sustain as AI assistants become increasingly useful and perceptive. Additionally, evaluators will lend more weight to attestations drawn from more comprehensive data, increasing the pressure to provide near-total access to AI assistants.

The pressure for AI attestations may extend to personal domains such as dating. Many people already consult AI agents for dating advice. If people come to view AIs as trusted judges of character, they may start requesting attestations from potential partners before agreeing to a date. Those who refuse would face a narrower pool of willing partners, extending the unraveling dynamic into intimate life.

Surveillance at the Civilizational Level

At this point, one might hope that data protection or anti-discrimination laws could prevent omnipresent observation. However, this is the insidiousness of self-surveillance; privacy legislation can help to prevent non-consensual surveillance by governments or businesses, but it cannot stop individuals from opting in to disclosure themselves. Unraveling can therefore happen anywhere.

Self-imposed surveillance can creep in under various political conditions. Different political systems will arrive at the same destination through different mechanisms: in the US and EU, the private sector will provide the infrastructure for surveillance. In China, the story is different: the government sidelined Zhima Credit because it wanted to control the infrastructure itself, evidence that surveillance in China may continue to trend toward mandatory top-down surveillance. The result may be the same: pervasive monitoring.

Conclusion

The future may involve substantially less privacy than the present. Throughout most of human history, privacy as we know it did not exist; our ancestors lived in small bands where reputation was built through direct mutual observation and gossip served as the original social credit system. The high-privacy society we inhabit today is a side effect of urbanization and the limited reach of pre-digital technology. As AI systems close that gap, we may be returning to the historical default. Societies tend to adopt values compatible with their technological constraints: the concept of intellectual property was meaningless before the printing press, and privacy may prove similarly contingent. Whether we accept this transition or resist it is an open question, but the forces driving it are already in motion.

While the direction of this trend seems robust, the form it takes is not predetermined. The question of who controls the AI attestation infrastructure matters enormously for how power is concentrated in the future: a world where attestation is managed by a handful of AI companies looks very different from one where it is controlled by governments, and different again from one built on decentralized protocols. Similarly, whether norms develop around narrow, domain-specific attestation or comprehensive behavioral transparency will determine how much power the system concentrates and in whose hands. These are important path dependencies, and the decisions shaping them are being made now.

Thanks to Dan Hendrycks and Devin Kim for formulating the premise of this piece.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Steven Veld is an AI strategy and governance researcher, with a particular focus on scenario-based forecasting for multipolar AI futures. He was a 2025 policy fellow with the Institute for AI Policy and Strategy (IAPS), where he worked on Congressional engagement with the AI Policy Network. Before that, he was a ML Alignment and Theory Scholars (MATS) fellow with the AI Futures Project. He has a BS in Computer Science at UCLA, and has prior experience working on biosecurity and compute governance.

In January, Anthropic CEO Dario Amodei published “The Adolescence of Technology,” an essay somberly assessing the risks posed by advanced AI. The day after, an influential WeChat account, AI Era, shared a breathless summary for its mainland Chinese audience: “Amodei warns that with AGI approaching, humanity is about to gain powers beyond imagination. But this power is also a sword of Damocles hanging over the heads of men...” AI Era’s summary was faithful, earnest, and engaged with Amodei’s essay on its own terms. It also describes Amodei as “gentle and elegant”—remarkably sympathetic treatment of one of the most vocal advocates of US chip export controls against China, and of an essay that describes the Chinese Communist Party (CCP) as an existential threat with a clear path to an “AI-enabled totalitarian nightmare.”

The pattern repeated in April, after US Senator Bernie Sanders hosted a panel on AI existential risk featuring leading Chinese academics Xue Lan and Yi Zeng. The event was picked up in a high-profile Chinese commentary syndicated across multiple sites, stressing Sanders’s concerns about existential risk and proposals for an international treaty similar to Cold War nuclear deals.

Amodei’s and Sanders’s treatment in the Chinese media landscape is not unique. As part of a seasonal fellowship with the Centre for the Governance of AI, I collected and analyzed over 60 mainland Chinese media sources discussing four recent Western essays that are relevant to the safety of advanced AI systems. I observed a pattern of urgent and credible Chinese summaries of Western AI safety work, though the coverage often avoided sensitive topics such as US-China competition or CCP control. Western AI safety authors have more impact on Chinese AI safety discussions than they may think.

How Western AI Safety Work Is Discussed in Chinese Online Media

The four essays for which I analyzed Chinese coverage of were “Situational Awareness,” by Leopold Aschenbrenner; “Superintelligence Strategy,” by Dan Hendrycks, Eric Schmidt, and Alexandr Wang; “AI 2027,” by Daniel Kokotajlo et al.; and the aforementioned “Adolescence of Technology,” by Dario Amodei.

All four pieces predict the emergence of superintelligent AI within the next few years. “Situational Awareness” argues for sustaining US dominance through an AI Manhattan Project. “Superintelligence Strategy” argues that states may mutually deter each other from developing superintelligence. “AI 2027” outlines a US-China race to superintelligence. “The Adolescence of Technology” outlines the key risks from “powerful AI” and how to defend against them.

Chinese coverage of the essays varied substantially. “Superintelligence Strategy” had the smallest footprint, with only six Chinese articles. “Situational Awareness” had 13 in total, including a second wave of coverage roughly a year after publication, driven by Aschenbrenner’s hedge fund returns. “AI 2027” and “The Adolescence of Technology” received the most, with over 20 articles each. The uptick over time suggests transformative AI is becoming more salient in China, perhaps partly driven by the “DeepSeek moment,” in early 2025.

Most sources received between 1,000 and 100,000 views, with only six exceeding 100,000. Total readership was likely over 1 million for “The Adolescence of Technology” and “AI 2027,” and in the hundreds of thousands for “Situational Awareness” and “Superintelligence Strategy.”

All four essays appear to be freely accessible in mainland China. “AI 2027” and “The Adolescence of Technology” were picked up by The Paper (澎湃新闻), a state media outlet. “Superintelligence Strategy” was discussed by a state-backed think tank. That was the extent of state-affiliated pickup. Major Chinese tech and financial outlets—Sina Finance, Xueqiu, 21st Century Business Herald, Wall Street News (华尔街见闻)—provided extensive coverage of all four.

The highest-quality sources were AI-focused WeChat accounts, particularly AI Era (新智元) and Synced (机器之心), each with millions of followers and regular syndication on mainstream sites. The highest-engagement discussions were on Zhihu (知乎), a leading Chinese Q&A website: threads discussing “AI 2027” reached nearly 200,000 views, while those discussing “The Adolescence of Technology” reached nearly 280,000.

The overwhelming pattern is one of serious, sympathetic engagement. Of the 61 primary sources, 85% adopted a neutral framing (i.e., they did not take a position on their content). One Zhihu user wrote a striking analogy: “We are now seconds before this car goes out of control. The people in the back of the car are still discussing where to eat later, completely unaware that the front of the car has already broken off the cliff.”

The Systematic Scrubbing of China

In the Chinese sources’ reporting on the essays, there is a consistent pattern of softening or omitting content that names China or the CCP as an adversary. Only 43% of the sources mentioned US-China competition at all, despite it being a central theme of all four essays. The rate varied sharply: 60% of “AI 2027” sources mentioned it, compared to just 18% of “Adolescence of Technology” sources. Where such competition was mentioned, it was almost always in passing.

The scrubbing operates on a spectrum from light to heavy. At the lighter end, a summary of “The Adolescence of Technology” from 硅星GenAI discusses “misuse for seizing power” in general terms, whereas Amodei explicitly wrote about the authoritarian threat from the Chinese government. A translation from a Chinese headhunting firm replaces “CCP” with “large state entities.” An AI Era post on “AI 2027” presented an abridged timeline that excluded all China-related discussion.

At the heavier end, Wall Street News, a prominent Chinese financial news and data provider, published a near-complete translation of “The Adolescence of Technology,” but it omitted the paragraph singling out the CCP as a threat. Everything else in Amodei’s article, including the general framework of risks from authoritarian misuse, remained intact.

Whose decision is the scrubbing? A LessWrong post on initial Chinese coverage of “AI 2027” argues that the sanitized posts likely result from self-censorship. Given that the original essays are freely available, and that some sources do deal with mentions of China head-on, the omissions are most likely choices by individual contributors or editors rather than the result of state instruction. But editorial decisions are also likely shaped by greater audience interest in the latest technological developments over geopolitical competition. This may also explain the more muted pickup of “Superintelligence Strategy,” which focuses on US-China deterrence: that topic is both more sensitive and less interesting to an audience focused on AI’s transformative potential.

A Window of Opportunity

These essays remain freely accessible, are covered by major tech and finance outlets, and are not systematically politicized. Those realities suggest that there is no coherent, state-led narrative around transformative AI in China at present, which gives Western ideas space to permeate the Chinese ecosystem.

Research by Matt Sheehan on Chinese AI regulation suggests a mechanism by which Chinese academics, journalists, and corporate researchers actively digest international AI debates and feed them into the regulatory process. The Chinese tech-literate commentariat—particularly contributors to outlets like AI Era and Synced—may have significant influence over policymakers and key employees at Chinese frontier-AI companies. For example, they may raise awareness that AGI might be developed soon and might be difficult to keep under human control.

This large, engaged Chinese audience shows that Western AI safety thinkers can inform Chinese audiences about their concerns, even without deliberate outreach. Such a pathway suggests a path to impact for Western AI safety work that is not widely discussed at the moment. Most directly, it presents a valuable opportunity to foster US-China cooperation on AI, by increasing mutual understanding of AI risks. But, even if international cooperation is undesirable or too difficult, there is value in raising Chinese awareness of alignment and loss of control risks.

It is good news that these ideas—including about the limits of human control—are widely and neutrally discussed in China, because they make it less likely that Chinese developers would build advanced AI systems while naive to loss of control risks. While this does not mean that Chinese AI systems will be safe, it does suggest a baseline awareness of AI risk.

These findings also have practical implications for international engagement. Track 1 and Track 2 dialogues can likely assume significant familiarity with Western conceptions of transformative AI—at least among tech-literate participants. Since they won’t need to spend time establishing basic threat models, these conversations may move more quickly to concrete governance mechanisms.

The signal is clear. Chinese tech commentators are reading Western AI safety essays, taking them seriously, and engaging with their ideas on the merits. The conversation is happening. Western thinkers should pay attention.

Thanks to my supervisor Oliver Guest, who provided the idea and detailed notes for this project, as well as to Emmie Hine, Karson Elmgren, Verena Heusser, Zilan Qian, and the 2026 winter fellows at GovAI who shared their comments.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Calvin Duff is a British diplomat currently on sabbatical to focus on AI governance. Most recently he was posted to Hong Kong, for which he trained for two years to achieve professional working proficiency in Cantonese and Standard Written Chinese. He previously served in Zimbabwe during the transition from Mugabe’s rule. Before the Foreign Office, he built neural networks for brain-computer interfaces at NASA’s Ames Research Facility. He recently completed a seasonal fellowship at GovAI, where he studied how transformative AI is discussed in Chinese online media.

In the three years since OpenAI launched ChatGPT, economists and AI researchers have published forecasts projecting that, over the next decade, AI will add to annual growth by amounts ranging from as little as 0.1% to as much as 30%. By 2035, the gap between these forecasts nears a quadrillion dollars: an amount that exceeds a decade’s worth of current global output.

The Quadrillion-Dollar Delta

Projected US GDP Under Alternative AI Growth Estimates, 2026-2035

Skeptical forecasts describe a future world barely different from today’s: one with modest productivity gains and manageable labor-market adjustments, all governed with incremental policy tweaks. Other forecasts describe civilizational transformation through explosive growth, wholesale economic restructuring, and unprecedented governance challenges. No treasury, central bank, nor legislature can prepare effectively for such a wide range of outcomes.

Yet the machinery of government does not pause for epistemic crises. Budgets are being written, pension obligations set, tax codes revised. Knowingly or not, each policymaking decision places an implicit wager on which future is most likely to materialize.

Betting on the wrong forecast for AI’s economic impact would carry severe consequences. If governments prepare for explosive growth that never arrives—borrowing against imagined AI windfalls to fund infrastructure and expand social programs—they risk sovereign debt crises and spiraling inflation when tax revenues disappoint. If they prepare for modest growth and explosive transformation arrives instead—maintaining current unemployment systems, underfunding retraining programs, and leaving regulatory frameworks unchanged—societies may face mass labor displacement, entrenched inequality, and unprecedented policy challenges, all of which can foster existential political instability.

This divergence in forecasts stems from three testable disagreements about how AI interacts with the economy: (1) whether AI’s jagged capability profile (excelling at some complex tasks while failing at seemingly simpler ones) represents temporary limitations or fundamental constraints on automation; (2) whether institutional friction limits technological potential, and (3) whether AI can automate innovation itself.

Instead of guessing about an AI-influenced future, we should analyze which assumptions are holding in the real economy. This article maps the three core questions driving the quadrillion-dollar delta—and shows what data policymakers should watch to better predict which world we’ll enter.

Question 1: Which Jobs Can AI Actually Automate?

The first disagreement sounds simple: what kinds of work can AI actually do?

MIT economist Daron Acemoglu projects that AI will boost US productivity by 0.71% over 10 years. University of Virginia economist Anton Korinek’s scenarios reach 18% annual GDP growth. Both Acemoglu and Korinek are leading scholars who published their estimates through the National Bureau of Economic Research, the most prestigious working-paper series in economics. Acemoglu won the Nobel Prize in 2024; in 2025, Korinek made TIME’s 2025 list of the 100 most influential people in AI. The 25-fold difference in their projections is not a disagreement about whether AI is impressive—each acknowledges that the technology is already impactful. It is a disagreement about whether AI’s current limitations are permanent, or will scale into the complex, context-dependent work where most economic value resides.

The Skeptical Case: AI Faces a “Hard Tasks” Wall

Acemoglu starts by focusing on the qualitative differences between tasks in the economy. An AI system that writes fluent marketing copy has automated a real job—but copywriters represent roughly 0.17% of the US workforce. However, the remaining 99.83% of jobs may still require skills that AIs are unable to do.

Health care workers—nurses, home health aides, medical assistants—represent roughly 18% of US GDP and 13% of total US employment. Combined with those serving in education and the skilled trades, these employees—whom Acemoglu calls “hard task” workers—constitute a commanding share of jobs and national output. In Acemoglu’s view, these jobs resist automation: they are context-dependent, socially embedded, and lack clear metrics for success.

Acemoglu taxonomizes tasks into “easy” and “hard” accordingly. Easy tasks have two properties: objective measures of success and a simple mapping between action and outcome. Computing a tax return, transcribing audio, standardizing a dataset—with such tasks, you know what “done well” looks like, and the steps to get there are straightforward. AI learns these quickly.

Hard tasks resist easy verification. Diagnosing a patient’s cough, hiring the right candidate, teaching a classroom of teenagers: the desired outcome depends on vast contextual factors, and whether the task was performed well may not be immediately evident. Without clear success criteria, AI can only mimic average human behavior rather than exceed it. Acemoglu argues that, even under optimistic assumptions about AI capabilities, AI exposure is limited to just 5% of output.

AI automation will displace labor, not replace it. As AI automates “easy” tasks, Acemoglu hypothesizes, labor won’t disappear; it will refocus on “hard” tasks toward which AI contributes little. In this model, even as call centers become more efficient and data processing accelerates, economy-wide productivity barely moves, because labor drifts into healthcare, education, and the skilled trades. In aggregate, the impact of AI on labor resembles squeezing a balloon: visible displacement but no overall contraction.

This is not hypothetical. It is exactly what happened during the first wave of personal computers. Spectacular technological capability produced only modest aggregate productivity gains, because the technology was applied to a narrow slice of actual work.

The Explosive Case: Difficult Tasks Can Be Surmounted

Korinek rejects this binary, arguing that task complexity is a distribution rather than a wall. Some tasks are easier, some are harder, but difficulty is a continuous spectrum: verifying an audio transcription is easier than verifying a medical diagnosis but much harder than verifying numerical calculations. While Acemoglu assumes that AI’s capacity to automate work will remain constrained to “easy” tasks for the foreseeable future, Korinek assumes that AIs will gradually climb this difficulty ladder along their existing trend.

This distinction is crucial. If task difficulty is a continuum and AI capabilities continue to scale with marginal compute and data—following the long-standing “scaling laws” observed in AI development—then “hard” tasks just require more scale. The question is not whether AI automates legal research or medical diagnosis, it’s when it will do so. The answer, history suggests, is sooner than most expect.

Labor-market data may illuminate AI’s emerging impact. The empirical battleground is already visible. Can AI handle the tasks Acemoglu classifies as structurally hard? Evidence is mixed and revealing. AI passes the US Medical Licensing and bar exams, each of which are supposedly hard tasks. Yet current AI and robotics systems struggle with physical tasks such as folding laundry and washing dishes—tasks any teenager can learn. More tellingly, adoption surveys show businesses hesitating to scale AI across their operations, revealing preferences about reliability that marketing materials obscure.

While AI benchmarks serve as proxies for the breadth of AI automation, this question will ultimately be settled in labor markets. If copywriter wages fall while nurses, teachers, and tradespeople remain valuable, Acemoglu is right: AI is automating “easy” tasks but not “hard” ones. But if wage gaps contract across occupations as AI adoption expands—faster than retraining and relicensing alone could explain—then the scaling hypothesis is eating through task complexity faster than skeptics believed possible.

Question 2: Can the Economy Absorb What AI Companies Produce?

AI capabilities are necessary but not sufficient for broad impact and automation. Even if AI can perform a task, deploying it at scale lies behind what technologists call “schlep”—the unglamorous work of implementation—such as navigating regulation, liability, and IT integration.

Goldman Sachs projects that AI will add 1.5% to annual productivity growth over a decade. The Organisation for Economic Co-operation and Development (OECD) models barely break 0.5%. Each organization acknowledges that AI performs useful work, yet they disagree about whether the economy can actually absorb that work at scale.

The Skeptical Case: Institutions Are Stickier Than Markets

OECD economists base their bearish forecast on economic friction. AI adoption is impeded by a thicket of regulations, licensing, liability, and inertia—all of which move at legislative, not technological, speed.

Consider health care, which represents roughly 10% of GDP in advanced economies. AI reads medical images more accurately than most radiologists. Yet deploying it requires navigating medical device regulations, malpractice liability, insurance reimbursement codes, and state credentialing boards. The EU’s AI Act classifies medical AI as “high-risk,” triggering extensive compliance requirements. US liability law falls short of adjudicating a new class of uncertainty: who pays when the algorithm misses a tumor? Medical boards move at the speed of committee consensus.

The result of this regulatory friction is spectacular lab performance, but glacial adoption.

The aggregate effect may resemble Baumol’s cost disease. Economist William Baumol showed how productivity-resistant sectors—health care, education, government—consume a growing share of GDP precisely because they resist automation. If AI accelerates manufacturing and IT but barely touches health care and education, aggregate productivity stays muted, because the economy keeps reallocating toward the resistant sectors. The economy optimizes the shrinking parts, while the growing parts stay stuck.

The Explosive Case: Markets Fund What Works

Goldman Sachs rejects the institutional friction narrative, treating AI as a General Purpose Technology (GPT) comparable to electricity or the combustion engine. The mechanism is capital deepening: firms see ROI, capital markets fund adoption, workers upskill, complementary technologies emerge, and productivity compounds.

The timeline for integrating other transformative technologies spanned decades: electrification took 40 years, while computerization took 20. Unlike past GPTs, AI may also accelerate its own adoption, automating compliance and streamlining regulatory approvals in ways that electricity and steam never could. If technology generates returns, capital clears obstacles. Regulatory frameworks adapt. Firms reorganize. What earns profits gets deployed.

The friction these models acknowledge is economic, not structural. The binding constraint on AI adoption is investment capacity and worker adjustment speed—not institutions.

While each predicts AI proliferation, these scenarios differ on how and when. The difference between the two cases is whether the binding constraint is technological or institutional. If friction dominates, the challenge is not R&D funding but regulatory reform and workforce retraining. The question shifts from “Can AI do the task?” to “Will we let it?”

The data to test this exists. Adoption surveys track implementation depth by sector and firm size. Regulatory approvals are public. The meaningful signal is not today’s adoption level but its trajectory—whether AI integration is accelerating beyond tech and financial services into regulated, labor-intensive sectors like health care. If it is, institutional barriers are proving permeable. If adoption plateaus in sectors where regulation, licensing, and liability bind, friction is dominating.

Question 3: Can AI Automate Innovation Itself?

Beyond disagreements over direct automation, economists also disagree over whether AIs can automate discovery: the key input that makes everything else in the economy possible.

The Skeptical Case: AI Will Provide a One-Time Economic Boost

Economists Martin Baily, Erik Brynjolfsson, and Anton Korinek’s 2023 survey provides the framework for understanding this position, even as the authors themselves argue for more transformative outcomes: AI is a General Purpose Technology (GPT)—broadly applicable, generating cross-sector spillovers, raising productivity of capital and labor simultaneously.

However, GPTs historically deliver what economists call a “level effect.” They make the economy permanently more productive, lifting GDP onto a higher path. They may also provide a modest “growth effect” by making innovation somewhat easier. But the long-run growth rate eventually returns to trend anchored by fundamentals, including the difficulty of discovery and the supply of human researchers.

History validates this framework for the GPTs of the past. Consider the Industrial Revolution, electrification, computerization—each produced massive one-time jumps in living standards, then growth rates settled. Individual GPTs make us permanently richer—but not permanently faster.

The Explosive Case: AI Will Produce Ideas

Economists Philippe Aghion and Simon Bunel, along with the Epoch AI team, ask a different question: what if AI automates not just the production of goods but the production of ideas?

Standard growth theory treats ideas as a special input to production. A drug design or chip architecture is non-rivalrous—once it’s discovered, everyone can use it simultaneously, without depletion. Long-run growth comes from accumulating these non-rivalrous ideas. More ideas, more growth.

AI might alleviate the human bottleneck on idea production. But idea production has always been constrained by the supply of human researchers. More scientists mean more discoveries. Better scientific tools—microscopes, computers—help, but ultimately humans do the research, and human researchers scale with population.

AI breaks this association. If AI systems can conduct literature reviews, generate hypotheses, design experiments, and interpret results, then idea production decouples from demography. The research workforce scales with compute, not population.

The mathematical consequences of AI generating ideas could be explosive. If the number of researchers grows rapidly because AI researchers can be produced by other AIs—and if that scale overwhelms the rate at which good ideas become harder to find— GDP growth becomes super-exponential. Year one: 2% growth. Year five: 8% growth. Year ten: 25% growth. Economists call this an “economic singularity.”

Acceleration in idea-production holds clues to AI’s true potential. Is AI accelerating the rate of discovery? Evidence is intriguing but far from conclusive.

AlphaFold’s protein structure predictions represent genuine acceleration in structural biology—problems that once took months now take hours. AI-designed drugs are going through clinical trials. Materials science uses ML to navigate vast design spaces that were previously intractable.

Yet patents per researcher in the United States—a rough proxy for idea productivity—show no obvious inflection. Scientific publications are becoming steadily less disruptive. Drug approval timelines have not shortened. Time from hypothesis to breakthrough in major fields remains stubbornly long.

The seeming lag could be attributable to measurement error, since scientific impact takes years to manifest in data. Or the lag could be real—AI is helping at the margins but not transforming the core issue, which is that hard problems are hard.

Empirical trends in science will reveal whether AI is accelerating ideation. Understanding AI’s impact on ideation is straightforward: watch science itself. If methods sections begin systematically crediting AI for hypothesis generation or experimental design, if patents per R&D dollar spent start rising, if time from research initiation to first patent filing compresses, if clinical trial success rates inflect upward, if venture-funded biotech suddenly posts dramatically higher hit rates, those findings suggest an active feedback loop. The key threshold is not whether AI assists researchers but whether research output begins scaling with compute investment rather than head count.

If scientific productivity continues its long-run decline, the standard pattern of GPT frameworks is likely to hold: we will get richer once, but we will not escape the gravity of diminishing returns.

Indicators to Resolve These Questions

The quadrillion-dollar delta is not an error term but a disagreement about mechanisms: task structure, institutional friction, and recursive innovation. These disagreements will not be resolved by better models or longer arguments. They will be resolved by reality.

Right now, the skeptics of an outsized impact of AI on GDP are winning on points. Meaningful adoption is slower than headlines suggest. Hard tasks remain hard. Institutional friction in industries like health care is significant. Data from the US Census Bureau shows AI implementation below 10% in most sectors. Occupational wage gaps are not compressing. Scientific productivity shows no upward inflection.

Skeptics may be right—but only for now. Those predicting explosive growth are not making a claim about 2026. They are making claims about trajectories—some betting that compute scaling will allow for greater task complexity, others arguing that capital markets will overwhelm institutional friction, still others envisioning that recursive innovation will compound the pace of scientific discovery. These trajectories diverge in mechanism and timeline, but all predict that the constraints currently limiting AI’s incursion into idea production will fall away. This absence of evidence is not evidence of absence, particularly for phenomena with long lag times.

Policymakers must work with the data they have. Leaders cannot wait for the debate to settle before acting, nor can they pretend to know which forecast is correct. But they can stop treating economic projections as prophecies to be believed or dismissed, and start treating them as hypotheses generating testable predictions.

The metrics proposed here are not the only relevant ones, but they may be sufficient to distinguish between the competing models. The Bureau of Labor Statistics tracks occupational wages. The Patent Office records filing timelines. Census surveys measure adoption depth. For these indicators, we do not have a data problem; we have an attention-allocation problem. The data is collected but not systematically analyzed as diagnostic indicators of which economic model is holding.

The quadrillion-dollar disagreement hinges on what metrics count most. It reflects the uncertainty about which economic mechanisms will determine outcomes, and which economic data to monitor. The correct policy response is not to average the extremes, pick a camp, or wait for certainty. It is to build monitoring infrastructure that tells us which mechanisms are actually operating in the economy, then adapt in real time as evidence accumulates.

The quadrillion-dollar disagreement will resolve in time. The question is whether policymakers will be watching when it does—and whether we will have built the state capacity to respond to whichever world we are entering.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Anton Shenk is an AI policy research associate at RAND specializing in the economics of artificial intelligence. His work examines how AI is reshaping markets, regulatory frameworks, and security risks.

In 1959, a midsize Swedish car company did something its competitors thought was myopic, if not reckless. It effectively open-sourced the three-point seat belt, the greatest safety innovation in automotive history. The prevailing industry wisdom at the time was blunt: “Safety doesn’t sell.” Just three years prior, Ford had offered seat belts for a $9 surcharge, as part of its 1956 Lifeguard campaign; despite Robert McNamara’s championing of the program, the safety push failed to give Ford a competitive edge. Henry Ford II reportedly grumbled, as he was dialing back its campaign, “McNamara is selling safety, but Chevrolet is selling cars.” But Volvo was neither myopic nor reckless; in fact, it saw further than any of its competitors. While they competed fiercely for dominance in a race for horsepower, engine efficiency, and design, Volvo could see that consumers cared about safety and reliability, too. The bet paid off: Volvo became one of the most recognized automotive brands in the world. According to Volvo, seat belts have since saved over 1 million lives.

American AI needs its Volvo moment.

The aim of catalytic regulation is to enable this moment. It is a family of positive incentives designed to channel market forces toward safety. Where traditional regulation works through mandates and penalties (“do this or else”), catalytic regulation works through rewards. Think tax credits for safety R&D, procurement incentives for verified-safe systems, and prestige mechanisms that make safety a competitive Schelling point. The goal is not just to subsidize safety at the AI industry’s margins, although that alone would be worthwhile. It is to catalyze a deeper shift in the culture that animates American AI innovation, marking safe and powerful AI as the very thing that American labs can do better than any competitor.

Let us now explore how catalytic regulation can meaningfully improve AI safety both today and in the future, before describing some specific example regulations.

Why Catalytic Regulation?

Why should we turn to catalytic regulation? For one thing, positive incentives are more realistic in today’s political environment, but they can also lay the groundwork for stronger regulations in future, while promoting better general safety culture within organizations.

Current deregulatory trends favor catalytic regulation. Any discussion of AI regulation must start by acknowledging political reality. In the United States, policymakers increasingly view AI through the lens of strategic competition with China. That outlook has produced a strongly deregulatory federal stance, grounded in the belief that limiting regulation is necessary to preserve innovation and US competitiveness relative to its international rivals. America’s AI Action Plan, which the White House released in July 2025, frames AI policy almost exclusively as a reaction to an AI “arms race,” calling for the curtailment of state-level regulation and the removal of barriers to innovation. Even relatively light-touch proposals face organized resistance from a tech lobby that has successfully cast any regulatory measure as saddling American developers with European-style burdens. For better or worse, there is currently relatively little organized political appetite for — and strong industry pressures against — traditional command-and-control regulatory measures such as mandates, prohibitions, and prescriptive safety requirements.

Catalytic regulation evades these barriers. Tax credits and procurement incentives neither impose compliance costs nor slow development. They do not ask senators to stand up and argue for restricting US companies. The pitch is simpler and politically commonsense: we are investing in US leadership toward safer AI. This strategy works with the grain of deregulatory attitudes. Indeed, as the AI race intensifies, the case for positive incentives only gets stronger, while the case for traditional regulation gets weaker.

Catalytic regulation complements other safety efforts. Catalytic regulation is not merely a second-best substitute for the regulation we wish we had. Suppose there was the political will to legislate AI safety: what standards or norms should we require labs to satisfy? There are already several reasonable ideas: it is not wise to release models that can contribute to the development of chemical and biological weapons; red-teaming can expose latent development gaps; and using models to perform behavior audits on other models can often improve robustness. But there is still much we do not know about how to make today’s models safe. For instance, studies have found that safety benchmarks are heavily correlated with model capabilities, meaning that capabilities progress can be misleadingly portrayed as safety improvements and making it harder for us to articulate clearer standards. We do not yet have the three-point seat belt for AI.

Here, the utility of catalytic regulation is that it can spur advances in our understanding of safety that are desperately needed to define any broader regulatory regime. If catalytic tools advance safety practices, benchmarking infrastructure, and institutional capacity today, they make future mandates more effective should the political winds shift. In this sense, catalytic regulation is also complemented by traditional regulation; while positive incentives alone may not ensure that all market participants adopt high safety standards, they can establish best practices as a standard of “reasonable care,” a powerful anchor in negligence law that regulators can point to when they do act. This pro-safety infrastructure, if built during the current regulatory drought, could become a foundation from which to manage the flood of AI disruption.

Catalytic regulation influences organizational culture. A broader benefit of catalytic regulation is the way it can influence “organizational culture” — the general attitudes that govern what people do “when no one is looking.” Organizational culture fosters values, illuminating the blind spots that regulators do not observe. It is what determines whether the chef washes their hands when visiting the bathroom, whether a NASA engineer stays overtime to check the numbers once more, and whether a nurse reports a surgery accident or closes ranks with the attending surgeon. Given the countless pockets of local knowledge that exist within AI labs (while remaining unknown to Washington regulators), inculcating safety as a value within organizational culture is key to minimizing future AI harms.

We can draw lessons from previous reliable organizations. The paradigmatic examples of organizations suffused by safety culture are High Reliability Organizations (HROs), which have long been studied by organizational sociologists. Familiar examples of HROs include nuclear facilities, air traffic controllers, and aircraft carriers. What ties them together is what physicist-turned-political-scientist G.I. Rochlin, in a prologue to New Challenges to Understanding Organizations (1993), calls their “effective management of innately risky technologies through organizational control of both hazard and probability.”

One surprising lesson from studying HROs is that their safety culture can precede safety regulation, rather than following it. After the Three Mile Island accident, in 1979, the nuclear industry created a self-regulatory body, the Institute of Nuclear Power Operations, to instill professional safety norms even before regulators codified them. Volvo made the three-point seat belt standard in 1959, nine years before federal law required seat belts in new passenger cars and decades before using three-point seat belts became a ubiquitous norm.

There is no master recipe for instilling safety culture, but catalytic regulation has some key ingredients. It builds on, and contributes to, an attitude of collaboration between society and the organization. It is designed not to punish deviation but to reward attention and care. All American labs have their internal “safety factions”; catalytic regulation aims to reward and amplify their work.

Catalytic regulation has positive expected value, with little downside. Even if it does not solve every safety problem, and it will not, the marginal question is straightforward: are we better off with these tools than without them? The measures are low-cost, politically feasible, and compatible with every other regulatory strategy on the table. They foreclose nothing while building the institutional scaffolding that any future approach will need.

From Principles to Practice: The Catalytic Regulation Bundle

If policymakers pursue catalytic regulations for AI, which specific interventions are most promising? We will discuss four potential mechanisms: corporate incentives, demand-side incentives, market guarantees, and prestige.

Corporate Incentives

The most direct form of catalytic regulation targets labs, the hubs of AI innovation, by offering tax subsidies for investments in AI safety research. In “Racing to Safety: Tax Policy for AI Safety-by-Design,” my co-author (tax expert Mirit Eyal) and I propose such a program.

The 1983 Orphan Drug Act provides a blueprint. The model here is the research and development tax credit, which offers firms a sizable incentive for investment in more basic forms of innovation. This credit has a long and successful history. One notable example is the Orphan Drug Act, which incentivized pharmaceutical companies to invest in finding life-saving interventions for rare diseases, left “orphan” by normal market forces. In the decades following the act’s passage, hundreds of orphan drugs received approval, creating an entire therapeutic category that the market had effectively abandoned.

The orphan drug credit did not solve a financing problem; pharmaceutical companies are already well resourced. Rather, it solved an allocation problem, redirecting where the next marginal dollar went. AI safety faces the same structural problem (with similar potential solutions). For instance, despite being flush with capital and publicly committing to devoting “20% of the compute we’ve secured to date” to its safety team, OpenAI later left the team with only 1% to 2%. Just as the orphan drug credit succeeded in directing the attention of highly capitalized pharma companies toward solving rare diseases, earmarked subsidies could incentivize the work of internal safety teams, especially given how low the baseline currently is. A safety R&D credit wouldn’t compete with a lab’s total budget; it would shift the calculus when a research director decided how to allocate the next team of engineers or the next cluster of GPUs.

These kinds of research subsidies also have the advantage of scalability. Policymakers could limit support if AI capabilities development stalls, or increase it if the pace of progress continues unabated. The most serious objection to safety research subsidies is the difficulty in distinguishing genuine safety investments from general capability development; as mentioned earlier, many safety benchmarks appear to track general model capabilities without labs focusing specifically on safety. This concern is real, and we design attestation and audit mechanisms around it in the companion paper. But labs’ incentives to invest in capabilities are already saturated; the possibility that companies might design elaborate schemes to redirect modest safety subsidies toward capability research would be economically irrational, especially in light of potential reputational blowback.

Demand-Side Incentives

While government regulations can shape the behavior of domestic corporations, AI safety is a global problem.

Even if the US were to implement regulations, jurisdictional and practical constraints would limit the direct effects on foreign developers, cloud providers, and other AI stakeholders. However, another form of catalytic regulations can help to bridge this gap; tax incentives directed at consumers, which offer rebates or credits for purchasing safe AI systems, can shape which products enter the US market, regardless of where those products originate.

US energy-efficiency standards for consumer products shaped global manufacturing. The energy-efficiency credit provides a model for a consumer-focused approach. When the federal government offered tax advantages for energy-efficient appliances, vehicles, and building systems, the immediate effect was domestic: American consumers shifted their buying habits toward qualifying products.

But the secondary effect was global. Foreign manufacturers seeking access to the American market adapted their product lines to meet American certification standards. Korean appliance manufacturers, Japanese automakers, and Chinese solar-panel producers all redesigned products to qualify. In this way, a domestic certification requirement effectively exported American energy standards without requiring any international agreement or diplomatic leverage.

The same model can foster AI safety adoption. The example of energy-efficiency incentives is directly applicable to AI. An “AI Safety Usage Credit,” providing a modest tax reduction for businesses and individuals subscribing to AI services that meet rigorous safety certification, would create a systematic price advantage for certified systems. A business choosing between two model providers would find that only one came with, say, a 10% tax credit per token. The certification would focus on properties that markets do not necessarily or fully price in: for example, robustness to adversarial inputs, resistance to deceptive alignment, interpretability of core reasoning, and safeguards against autonomous misuse. We may not yet have a comprehensive set of criteria for AI systems to meet, but we could start with common sense certification requirements, and update them on a semi-annual or annual basis, as safety innovation flourished.

The international implications are significant, because consumption subsidies give us leverage over labs that reside outside of our jurisdiction, potentially catalyzing a much broader cultural change in AI development.

Market Guarantees

Tax incentives can mitigate the cost to developers of investing in safe AI. However, they do not fully eliminate the “appropriability problem”: firms that share breakthrough safety techniques bear the development costs while competitors freely implement their discoveries. Insofar as safety and reliability are a market advantage in their own right, companies are incentivized not to share their advances even if doing so would benefit the public. To address this challenge, we can turn to a different form of catalytic regulation: market guarantees.

Government spending influences private-sector decision-making. A big part of Operation Warp Speed — which used federal dollars to incentivize private-sector COVID-19 vaccine development and safety assessment — consisted of using the federal government’s purchasing power as a regulatory lever. Similarly, we could develop an “AI Safety Charter” program to give firms demonstrating safety leadership a bundle of concrete benefits: priority consideration in federal procurement, expedited security-clearance processes, streamlined regulatory compliance, and access to government datasets for safety research. As illustrated by the recent face-off between Anthropic and the Department of War, federal purchasing can be incredibly consequential even for market-leading AI labs.

Patents can regularize safety proliferation. The patent system can address the appropriability problem more directly. A lab that published a breakthrough interpretability method cannot prevent competitors from adopting it, yet the social interest in rapid diffusion conflicts with the private interest in maintaining a competitive advantage. Mandatory “fair, reasonable, and non-discriminatory” (FRAND) licensing at modest royalty rates could give innovators a period of competitive advantage without bottlenecking the diffusion of safety knowledge.

The Power of Prestige

The mechanisms above operate through material incentives. But material incentives address only part of what drives organizational behavior. Status competition — the pursuit of professional standing relative to others — is at least as powerful a motivator, and, unlike with tax credits, its effects tend to persist after the initial intervention.

Federal recognition can bring market benefits. Governments already harness companies’ desire for prestige to shape their behavior, sometimes to striking effect. The Malcolm Baldrige National Quality Award, created in 1987, to promote “improved quality of goods and services” in the US, carries no cash prize. Yet an analysis commissioned by the National Institute of Standards and Technology estimated a benefit-to-cost ratio of roughly 820 to 1, with gains flowing primarily from process improvements that firms undertook in pursuit of recognition. The Occupational Safety and Health Administration’s (OSHA) Voluntary Protection Programs (VPP) tell a similar story: participants receive a worksite flag indicating that they meet VPP standards, and a listing on the OSHA website. Moreover, they directly see the benefits of adhering to the standards, seeing injury rates approximately 50% below industry averages. These programs work because visible honors shift what good engineers optimize for.

The AI industry appears well-positioned for such interventions. Researchers already compete intensely for conference acceptances, citation counts, and benchmark rankings. Labs recruit by advertising leaderboard positions. A “Presidential Frontier AI Safety Medal,” modeled on the Baldrige Award, would recognize breakthrough safety research through White House ceremonies, with deliberate scarcity (honoring perhaps three organizations annually) and a requirement that recipients publish certain safety practices or tools. A public safety leaderboard maintained by an attestation body would track which organizations solve recognized safety challenges first, harnessing the competitive instincts that currently drive benchmark races.

Prestige cannot be easily gamed. As a part of catalytic regulation, prestige incentives could help fill some of the gaps left over by financial incentives alone. One of those is “gameability.” Tax credits can, in principle, be gamed; prestige cannot be reliably fabricated. A lab cannot consistently fake the technical achievement required to top a public leaderboard or win peer recognition. If prestige incentives are well-recognized and hard to game, they might meaningfully shift research incentives. Companies may also see prestige from safety work as a way of attracting top talent, a scarce resource for which they are already competing ferociously.

Ultimately, catalytic regulation aims not to outspend the labs but to shift the competitive equilibrium. Financial incentives provide the activation energy, but the bigger, deeper, and more sustainable impact will come from cultural and competitive dynamics. When leading labs compete visibly on safety, they set norms that smaller labs adopt too — not because regulations require it but because the organizations they aspire to emulate have made safety a core part of what excellence looks like. The same competitive dynamics that accelerate risk-taking in the current capabilities race can, with modest institutional design, be redirected toward safety leadership.

A Seat Belt for AI

Regulation is hard, especially for an unknowable future. Yet, during a period of intense skepticism toward traditional regulatory measures, catalytic regulation offers a path forward. At its best, it redefines the terms of the AI race, positioning safety as a non-negotiable reputational priority for American labs, and showing that safety guarantees make their systems more attractive to governments, enterprises, and end users worldwide. Even at its most modest, catalytic regulation still spurs safety advancements that seed the ground for any regulatory architecture that comes next. In 1959, Volvo gave away the seat belt and led the world. The question is whether US policymakers will recognize the same opportunity for American innovation, and whether they will do so before another nation does.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Yonathan Arbel is the William Alfred Rose Professor of Law at the University of Alabama, and the Director of its AI Legal Studies Initiative. He earned his doctoral degree in law and economics at Harvard Law School, where he was both an Olin fellow and a Byse fellow. He is serving as a co-director of CLAIR, the Center for Law & AI Risk.

Last December, President Trump announced that the United States would allow Nvidia to sell its powerful H200 AI processors to customers in China. Officials in the Trump administration have long argued that the best way to win the AI race is to promote the export of US technology around the world, not to restrict it. Selling H200s, the administration claims, will boost the market share of US chip-makers while preserving the US hardware lead.

Critics warn of national security risks to selling advanced chips, but the administration appears committed to its course. Taking a pro-export framework as a given, policymakers can balance between market share and national security by managing a quantity that currently doesn’t receive enough attention: the United States’ relative compute advantage. Total AI compute for a country is calculated as its stock of AI processors weighted by the effectiveness of each processor; the US currently enjoys a roughly 10-to-1 compute advantage over China. That advantage matters because more compute can support more domestic R&D, more customers served by American AI products, and more ability for the US government to influence the development and use of AI.

Relative compute advantage is about the quantity of chips as much as the quality. Administration officials emphasize that the H200 has been superseded by Nvidia’s more powerful Blackwell generation, but many of the world’s largest AI supercomputers still use H200s. With enough of them, Chinese developers may be able to train and deploy AI models that are competitive with US models. Chinese companies have reportedly placed orders for more than 2 million H200s already.

Whereas Biden-era export controls attempted to make the US compute advantage as large as possible, an export-friendly framework could instead focus on maintaining a fixed, favorable compute advantage. To do so, policymakers should peg the quality of exported chips to the performance of China’s domestically manufactured alternatives, while capping quantities of those US chip exports.

A Return to Relative Advantage

Before the US government launched its all-out chip war with China, in 2022, American policymakers generally sought to maintain a one-to-two-generation advantage in key technologies. When the first Trump administration restricted sales to Chinese chip producers, in 2018, for example, it cut off only a limited set of the most recent semiconductor manufacturing equipment, and placed targeted restrictions on specific companies seen as national security threats.

The United States abandoned relative advantage in 2022. When the Biden administration imposed sweeping new controls in 2022, it declared that a policy of relative advantage was no longer viable. Then-National Security Advisor Jake Sullivan explained that the United States would henceforth seek to “maintain as large of a lead as possible” in certain foundational technologies, including advanced logic and memory chips. In practice, that meant setting a fixed threshold for chip performance, one that the administration progressively lowered, curtailing an expanding swath of China’s chip imports. At the same time, the Biden administration attempted to degrade China’s ability to produce rival chips by imposing increasingly draconian restrictions on semiconductor manufacturing equipment.

A revival of the relative framework. The second Trump administration has signaled a return to the logic of relative advantage. Rather than impose a general blockade on chip exports above a fixed level, the administration is reportedly planning to sell chips that are “about 18 months behind the state of the art.” Earlier last year, when the Trump administration approved exports of Nvidia’s inference-focused H20 chip, Commerce Secretary Howard Lutnick defended the policy of selling Nvidia’s “fourth-best” chip to China.

A widening hardware-performance gap. Yet critics point out that US-made chips are so far ahead of Chinese chips that even chips well behind the US frontier are better than anything China can make. Compared with China’s best chip (the Huawei 910C), the H200 has about 32% greater processing power and 50% more memory bandwidth. Huawei forecasts that its future offerings will improve more slowly than Nvidia’s, so that gap will only grow.

‍Current export controls don’t guard the US relative compute advantage. In January, the Trump administration took a step toward managing the volume of chip exports, releasing rules that would allow Chinese customers to buy up to half as many H200s as are sold in the United States. But the rule could still allow as many as 2 million sales. The current ratio between US and Chinese compute is likely somewhere between 9 to 1 and 12 to 1, so allowing China sales equivalent to 50% of US sales would give Chinese companies a major boost compared with where they are today.

The current approach, then, has two gaps. First, it runs the risk of significantly raising Chinese AI capabilities, by allowing sales of chips whose capability reflects US chipmaking progress rather than China’s domestic chip quality. And, second, its limitations on sales volume, while a step in the right direction, are probably too generous to China. The administration can do better on both fronts.

Selling Chips Without Selling Out

When deciding which chips to approve for sale in China, the administration should peg export approvals to the performance of Chinese chips, not the US frontier. It should allow chip sales matching the performance of China’s latest widely available domestic offerings. That way, US chip designers will be able to compete with producers like Huawei in the Chinese market, but Chinese developers will not automatically reap the benefits of faster US progress in chip design. Considering only chips that China can produce in commercially relevant quantities — not one-off high-performance prototypes — would guard against efforts by Chinese companies to game the system.

On the question of how many chips to allow, the strategic importance of compute is a strong point in favor of restraining not only China’s access to the very best chips but also its ability to amass large quantities of sub-frontier-quality hardware — chips that could power military systems, intelligence analyses, and industrial robotics. Ultimately, the US should look to relative compute advantage to decide the quality and quantity of chips that it exports.

Selecting a target compute advantage. It is a positive sign that the administration has adopted a ratio approach by capping Chinese sales of relevant chips, but its 2 to 1 ratio is probably too generous. Aiming for a relative compute advantage a bit below the current ratio — perhaps around 8 or 9 to 1 — could still allow significant chip exports without giving China such a large step up. If the 50% ratio is applied to future approvals of more powerful chips, the gap will narrow even further. Over time, the administration should tighten its rules, especially if it considers future, more powerful chips for export. That way, it may be able to keep some Chinese developers in the US chip ecosystem while avoiding a free-for-all that could turbocharge Chinese AI at the expense of US labs and startups.

A more stringent approach was recently put forward by US Rep. John Moolenaar (R-MI), who chairs the House Select Committee on the Strategic Competition Between the United States and the Chinese Communist Party. Moolenaar argues that US officials should approve exports only when total US AI data-center compute exceeds Chinese compute by at least 10 to 1. If the ratio dips below that level, the US would cut off sales; if the ratio rises above it, the chips could flow again.

Making Relative Advantage Work in Practice

Shifting export-control policy to focus on the overall compute balance will bring practical difficulties. Just as the Biden administration struggled to set the right technical thresholds for its controls on chips and semiconductor manufacturing equipment, so the Trump administration will find it tricky to agree on a given compute ratio. Any such figure will need to be informed by an assessment of China’s existing AI computing power, which is difficult to ascertain.

Analysts will have to estimate, among other things, China’s existing installed chip base, the production capacity of Chinese fabs, the volume of legal imports, the smuggling rate, and the lifespan of different kinds of chips. There is already serious public disagreement about some of these figures: the Trump administration estimated last year that Huawei could produce 200,000 of its Ascend AI chips in 2025, while the outside group SemiAnalysis put the figure at about 800,000. Estimates of the rate of smuggling are similarly uncertain.

The trouble with yearly estimates. One suggestion, advanced by Moolenaar, is for the intelligence community to produce a yearly estimate of the US compute advantage over China. But US spy agencies are not geared toward this kind of commercial intelligence. Furthermore, updating the number just once each year will magnify the effect of any errors: if the estimate is wrong in either direction, the US government will approve or block a year’s worth of orders based on faulty information. Because chips last for several years, any mistakes will have ramifications well after the estimates are corrected. DeepSeek’s 2025 models, for example, were likely trained in part on Nvidia chips imported before updated controls went into place in 2023.

Which way to err. These are thorny but not insoluble issues. The US government can rely on both intelligence and external industry sources to estimate Chinese production. Variables like chip lifespan and smuggling rates can be known with at least some degree of confidence. The real trick will be choosing how to be wrong: if the biggest risk is underestimating Chinese progress and allowing too many chips to flow, policymakers should pick a conservative ratio. If instead the true risk is hurting American companies that could be selling to China — without compromising national security — they should err on the side of allowing greater sales.

The Trump administration has committed to selling powerful US AI hardware. That approach risks exporting increasingly advanced chips in essentially unlimited numbers — a surefire way to erase one of the main US advantages in the AI competition. Relying on compute ratio is not a perfect answer, but it offers the best chance of preserving the US advantage in AI while carving out more flexibility for industry to sell to China.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Alasdair Phillips-Robins is a fellow in the Technology and International Affairs Program at the Carnegie Endowment for International Peace, where his research focuses on emerging technology and national security. From 2023 to 2025, he served as a senior policy advisor to U.S. Secretary of Commerce Gina Raimondo, where he covered AI, semiconductors, export controls, and other emerging technology and international issues.

Noah Tan is the James C. Gaither Junior Fellow for the Technology and International Affairs Program at the Carnegie Endowment for International Peace. His work centers on AI supply chains and international technology competition. Previously, he was a research affiliate at Stanford’s Center for International Security and Cooperation and the Hoover Institution, where he worked on international security and economic statecraft. He holds a B.A. In International Relations with Honors and Distinction from Stanford University and is a 2027 Schwarzman Scholar.

In late 2025, Figure AI placed its third-generation humanoid robot into real homes for alpha testing. The Figure 03 has hands with 16 degrees of freedom, tactile sensors that detect forces as small as three grams, and foam-padded limbs designed for safe operation around people. It charges wirelessly and responds to natural language commands. It can learn in real time, adapting to its environment.

A few months earlier, Unitree started shipping the R1, a home-capable robot priced at $4,900. TIME included it among the Best Inventions of 2025. You can order one today and have it in weeks: it’s the most commercially accessible humanoid robot on the planet.

These robots have graduated from prototyping. They’re consumer products with price tags, shipping dates, and marketing campaigns. It’s easy to imagine a world in which every family relies on one or several robots to conduct daily life, especially as AI becomes more capable. But what rules govern a learning, physically capable, always-on AI device operating inside someone’s home?

Unfortunately, we’re far from a coherent answer. Existing US regulations were developed with Roombas and robot arms in mind, not autonomous humanoids, resulting in a confusing patchwork of obligations. That doesn’t mean the situation is hopeless, just that regulators must act quickly to establish reasonable standards for a generational technology. That work should start now, and not after the first serious home-robot injury, not after a data breach exposes 3D maps of thousands of homes, and not after a liability lawsuit reveals that no one can legally be held responsible.

What Robotics In 2030 May Look Like

To understand just how unprepared we are to govern next-generation robots, consider the trajectory that robotics development and adoption is taking.

Robotics is a rapidly expanding market. Goldman Sachs revised the total addressable market for humanoid robots to $38 billion by 2035. This is a sixfold increase from its prior estimate. The companies behind these projections are well-capitalized and have real products. Figure AI has raised over $1.75 billion at a $39 billion valuation. Boston Dynamics and Agility Robotics are already piloting commercial deployments at automotive and logistics facilities.

Robotics foundation models are improving rapidly. The technical capability curve matters as much as the business investment. Foundation models such as Vision-Language-Action (VLA) models now enable robots to respond to vague instructions, generalize across tasks, and make autonomous decisions in situations their programmers never anticipated or designed them for. Nvidia’s GR00T N models and Figure’s Helix represent a new generation of robot intelligence that can improve across frequent updates.

Household integration is on the horizon. By 2030, the realistic picture looks something like this: tens of thousands of humanoid robots operating in factories and warehouses, with the first wave of consumer-grade units entering homes. They’ll fold laundry, load dishwashers, assist elderly family members, and navigate living rooms alongside children and pets. They’ll be connected to the internet, continuously collecting visual, audio, and spatial data. And they’ll be governed by a regulatory patchwork that was never designed for them.

The ceiling of what the technology will ultimately achieve is debatable, but what matters is that meaningful numbers of these devices will populate homes within the next few years, yet the regulatory framework consumers need is far behind what they or their policymakers realize.

The Governance Gap

Consider the challenge facing a compliance officer at a company that’s preparing to bring an AI-enabled home robot to the US market. The robot has arms, hands, cameras, and microphones, and it runs a foundation model. What does the regulatory landscape look like for such a product?

Unfortunately, it looks fragmented, with numerous overlaps and gaps.

Outdated consumer safety frameworks. The compliance officer’s first stop is the Consumer Product Safety Commission (CPSC), the primary federal agency with authority over consumer product safety. Its frameworks are intended for products that remain substantially unchanged after leaving the factory. Yet the CPSC’s own reporting acknowledges that AI-embedded products learn from consumers after purchase, making traditional premarket testing insufficient and post-purchase hazards unpredictable. CPSC professionals have been aware of this problem for years. The agency hosts forums on AI-enabled products and previews pivots toward AI-driven hazard detection, but it has not finalized any mandatory standard for AI-enabled consumer robots.

Existing privacy laws are ill-suited for robots. Now consider the added privacy concerns. An always-on robot with cameras, microphones, and navigation sensors in a private residence creates a data-collection profile unlike anything current privacy laws were designed to address. California’s Consumer Privacy Act (CCPA) and Privacy Rights Act (CPRA) trigger extensive obligations around visual, audio, and biometric data. Illinois’s Biometric Information Privacy Act (BIPA) requires informed written consent before collecting biometric identifiers, with statutory damages of $1,000 to $5,000 per violation. However, BIPA was designed to govern apps and kiosks, not autonomous machines in private homes.

Similarly, the Federal Trade Commission’s (FTC) Children’s Online Privacy Protection Rule requires verifiable parental consent for passive data collection from children under 13. This creates a compliance challenge that borders on the absurd for an always-on device that can’t reliably distinguish between an adult’s face and a child’s face in every moment of operation.

Fragmented state AI regulations. A company selling nationwide must navigate 15 (and counting) different states’ privacy frameworks, none of which were designed for continuous visual and audio monitoring, spatial mapping that creates detailed 3D models of homes, behavioral pattern analysis, or consent challenges when guests enter the home.

For example, Colorado’s AI Act, effective June 30, 2026, requires developers of high-risk AI systems to implement risk-management programs and conduct impact assessments. California’s Transparency in Frontier AI Act became effective January 1, 2026. Texas signed its Responsible Artificial Intelligence Governance Act (TRAIGA) in June 2025. Each has different requirements, definitions, and enforcement mechanisms. Meanwhile, the federal government sends mixed signals. President Trump’s Executive Order 14365 established a Justice Department task force to challenge state AI laws, but an attempted 10-year moratorium on state AI laws was defeated in the Senate.

The industry suffers from unclear product liability. Perhaps the most consequential gap is in product liability. When a home robot causes an injury, who’s liable? The hardware manufacturer? The foundation model developer? The system integrator? The robot’s owner, who gave a vague command? Existing product-liability doctrines assume clear human oversight and products that behave predictably. AI systems with varying degrees of autonomy break both assumptions. Some legal scholars have proposed tiered approaches, but these require voluntary safety frameworks to exist in mature form; they don’t, for embodied AI.

Incomplete technical standards. The standards landscape reflects the same pattern of partial coverage. ISO 13482, updated in 2025, addresses personal-care robots but predates modern AI capabilities like foundation models. ISO 25785-1, the first international safety standard for bipedal robots, covers only industrial workplace use. IEC 62443 applies to networked industrial robots from a cybersecurity perspective but wasn’t designed for always-on consumer devices in private spaces.

Overlapping jurisdictions complicate matters. No single agency has comprehensive authority. And no single framework addresses the full range of risks, including physical safety, data privacy, AI decision-making, and cybersecurity. The gaps between these overlapping regulatory spaces are precisely where the real risks live. The compliance burden reflects the genuine complexity of a product that is simultaneously a consumer device, a physically capable machine, an autonomous AI system, and an always-on data collector.

Proven regulatory templates do exist. However, the situation is far from hopeless, with an established playbook of regulation for emerging technology that can inspire effective regulation for robotics. The National Institute of Science and Technology’s (NIST) AI Risk Management Framework is the most globally influential risk-management framework for AI. The Food and Drug Administration’s (FDA) Software as a Medical Device guidelines demonstrate that life-cycle-based AI regulation for physical products is feasible, while its Total Product Life Cycle approach recognizes that AI software evolves over time, with predetermined change control plans that allow manufacturers to get preapproval for categories of changes. The Federal Aviation Administration’s (FAA) graduated, risk-based approach to drone regulation provides a successful template for governing autonomous physical systems. The US can apply what it already does well to the specific challenge of embodied AI.

What Should the US Do?

So what would coherent US governance for embodied AI actually look like? What follows is not a finished proposal. However, it does detail the foundation that any serious approach would need.

Risk-tiering by physical capability and autonomy level. Not every robot presents the same risk, and each level should carry proportionate requirements. Reasonable tiers include:

• Tier 1. Stationary AI assistants (smart speakers, fixed cameras): low risk, where existing Internet of Things frameworks largely suffice.

• Tier 2. Mobile robots without autonomous limb manipulation (vacuum robots, delivery robots): moderate risk.

• Tier 3. Mobile robots with autonomous limb manipulation (home robots with arms and hands): high risk, given always-on sensors and autonomous physical decision-making.

• Tier 4. Humanoid robots in care settings (elder care, child care, medical care): highest risk, given proximity to vulnerable populations and life-safety decisions.

Purpose-built data governance. Privacy laws like California’s CCPA/CPRA and Illinois’s BIPA were designed for websites and apps, not physical devices operating in shared spaces. Their core principles (like user consent, data minimization, and the right to deletion) are sound, but need to be adapted for robots that move through homes and workplaces, collect data from multiple people simultaneously, and operate continuously.

Tiered liability. Compliance with established safety frameworks should be rewarded with a negligence standard for liability, while noncompliance should trigger strict liability. That creates the right incentive structure, but it works only if we define what “compliance” means for each risk tier.

Incident reporting. The US has proven templates for effective incident reporting. The Occupational Safety and Health Administration (OSHA) requires work-related fatalities to be reported within 8 hours; the Cybersecurity and Infrastructure Security Agency’s Cyber Incident Reporting for Critical Infrastructure Act requires that significant cyber incidents be reported within 72 hours; the FDA’s MedWatch system captures device malfunctions. An embodied AI reporting framework would need to cover physical injuries, near misses, privacy violations, unexpected autonomous behaviors, and cybersecurity incidents. But, under current law, it’s unclear which agency should receive, analyze, and act on such reports.

Premarket assessment. The FDA’s Total Product Life Cycle approach, with its predetermined change control plans for learning software, offers a directly transferable model. Robots with preapproved change control plans for AI model updates would address the fundamental challenge that AI-enabled products evolve after sale.

Coordination. CPSC, FTC, NIST, OSHA, state attorneys general, and state privacy regulators all have partial jurisdiction; none has comprehensive authority. Whether through designating a lead agency or creating something like a “National Robotics Safety Board,” there needs to be a single point of accountability.

Proactive standards development. The FAA’s experience with drones is instructive. FAA Part 107 created workable rules that enabled the commercial drone industry to grow while maintaining safety. Proactive engagement now could compress the typical five-to-seven-year regulatory stabilization timeline and avoid the reactive, incident-driven pattern that has characterized previous rounds of safety regulation, endangering consumers.

Conclusion

None of this is easy. A risk-tiering approach requires drawing lines that will feel arbitrary at the margins. Unified data-governance standards could slow time to market. Premarket assessments add costs that could disadvantage US companies against international competitors facing less regulatory pressure. The current administration’s deregulatory posture makes new federal frameworks politically unlikely in the near term.

But the products are coming, regardless of whether the governance is ready. The question isn’t whether embodied AI needs governance; it’s whether we build it proactively or reactively, after the incidents that make national headlines.

Many building blocks for meaningful regulation already exist. What’s missing is their integration into a coherent framework purpose-built for products that are simultaneously consumer devices, physically capable machines, autonomous AI decision-makers, and always-on data collectors in private spaces. That work should begin now, while there is still time to get the regulatory architecture right.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Tristan Ingold is an AI governance professional with a background in information security, risk, and compliance. He has led regulatory and commercial audits within large technology enterprises, and his current work focuses on the governance challenges emerging at the intersection of Frontier AI and enterprise risk. Tristan is a guest lecturer on cybersecurity assurance practices at the University of Washington and has authored two LinkedIn Learning courses on governance, risk, and compliance, reaching over 75,000 students to date.

Last week, Jack Dorsey, a co-founder of Twitter, announced that his company Block is cutting its head count from 10,000 to fewer than 6,000 because AI tools mean it needs fewer workers. It is not the first company to make such an announcement, and won’t be the last. But it raises a question: if AI takes jobs, are workers doomed? To many observers, the answer must be yes. Negative consequences for the labor force seem like an inevitable byproduct of advancing AI. Indeed, machines that automate work seem to promise exactly such outcomes: they start performing tasks that labor once did, which seems to imply that workers will experience worse economic prospects. Faced with the possibility of being displaced, many might hope that AI progress will slow or even stall, allowing humans to remain competitive.

But, in evaluating AI’s implications for the workforce, it’s misleading to think of AI as simply a replacement for labor. To complete the picture, we need to consider the fuller set of forces that are unleashed when machines automate types of work. In particular, we must engage two other economic features that sharply condition outcomes for labor: first, how machines affect prices, and second, the role of economic bottlenecks. Analyzing these dimensions leads us to a counterintuitive notion: the labor side of the economy will be far better off economically if AI is much, much better than humans at the jobs it replaces. In other words, rather than rooting for AI to fail, we should hope it exceeds us by a large margin.

Two clarifications before we begin. First, this argument does not assume a near- or mid-term future in which AI replaces all human labor (though we will discuss that possibility at the article’s end). It assumes uneven technological progress, in which dramatic productivity gains appear in some domains — and perhaps very many — while others remain more dependent on human labor. Second, when evaluating outcomes for workers, even if average living standards rise substantially, the transition can bring significant disruption, as well as real hardship for some workers. We will discuss this key issue as well.

Before turning to the mechanisms that drive value toward labor, it is useful to ground the discussion in history — specifically, the rise of agricultural machinery and the emergence of computers.

Automating Agriculture and Computing

Automation caused a huge drop in the percentage of workers employed in agriculture. In the United States today, less than 2% of the workforce is engaged in agriculture. But back in the 18th century, agricultural employment occupied 80% to 90% of the US workforce — i.e., almost everybody. This shift from the large majority of workers being farmers to a tiny minority being farmers is typical among leading economies, including France, Germany, Italy, Spain, the UK, and Japan.

So let’s imagine you traveled back in time to colonial New England or rural France and told farm laborers: “Look, machines are coming that will replace almost all of your jobs. They’ll plant, water, fertilize, harvest, thresh the wheat, shuck the corn, mill the grains…” You might mention a specific machine called a “combine harvester” that in the year 2001 would process 1 million pounds of corn in eight hours. To the extent the farmers believed you, they might naturally have two immediate reactions. First, they would expect farm labor (i.e., pretty much all labor at the time) to become impoverished as the machines took over nearly all of their jobs. Second, and related, they might anticipate that those who owned the farming machines would capture most of the income.

But history unfolded in a strikingly different way, with workers ultimately sharing in unprecedented prosperity. Real income per worker in the US today is about 25 times larger than it was in the late 18th century. Labor continues to capture about two-thirds of total national income. And, while companies like John Deere do very well making and selling combine harvesters and other amazing agricultural machines, their market capitalizations and sales are tiny portions of the overall economy.

Now, AI might not trace the same path as agriculture. But the farming experience should at least open one’s mind to the idea that “machines replace labor” is not the whole economic story, and that it is possible to destroy most of the jobs in an economy and see workforces thrive over the long run.

Computers have become ubiquitous, but they receive only a small fraction of spending. A more current example is computing itself. Over the past 50 years, we have witnessed mind-boggling advances in computer productivity that outpace human capabilities at an ever-expanding array of tasks. Computer use has expanded from niche applications on expensive mainframes, in the 1960s, to ubiquitous devices seen at work, at home, in our pockets, and in machines throughout the economy. Thanks to their ever-expanding capabilities, computers have replaced or substantially infiltrated many types of work — including clerical, secretarial, and manufacturing labor. Reflect for a moment on your “screen time” as a measure of how pervasive computers have become in our lives.

Yet, despite this expansion, expenditures on computing remain modest: business investment in computer equipment and software now peaks at around 4% of GDP. Similarly, households devote tiny shares of their budgets to computers and software — we spend many multiples more on housing, transportation, health care, or even restaurant meals.

How can it be that computers have taken over our lives, and yet we spend such small shares of our income on them? And how did farming machinery replace the vast majority of jobs in the economy, even as labor continued to thrive? To help answer these questions, we need to think about how these machines affect prices and about the knock-on effects for how value is distributed across economic sectors.

Machines and Prices

Computers are so efficient that the price of computation is close to zero. Computers represent a small share of the economy because, although they perform a huge quantity of tasks, they do so at very low cost. The result is that the services they provide become very inexpensive. In fact, many computer services are provided to consumers virtually for free. Housing is not free. Haircuts are not free. But internet search is effectively free. Offering things for free is partly about business models, but it is largely about how efficient the technology is. For example, an average human can do a simple mathematical calculation — say multiplying two four-digit numbers — in about one minute. My smartphone can do about 500 billion such calculations in one minute. Comparing the median wage to the cost of electricity, my smartphone is also about 5,000 times less expensive than a person over that one minute. So computers make multiplication virtually free.

The better a machine is at automating a task, the lower it will drive prices for consumers. This brings us to the first, major edit to the idea that machines replacing labor is necessarily bad for the labor force. Because automation doesn’t just replace work. It also makes output cheaper. In fact, that is the exact reason we deploy automation in the first place. Businesses, in pursuit of profits, generally seek to produce their goods and services at lower cost, so they use whichever approach — with whatever combination of machines or humans — they believe to be cheapest.¹ On the other side, consumers typically prefer lower prices for a given product or service, driving demand toward the firms that are more cost-efficient. Indeed, widespread discontent over inflation in the US reminds us that people care a whole lot about prices, not just jobs, when assessing their standard of living.

If AI takes some jobs, let’s hope it excels at them. Following this logic, we can see that the more productive the machine, the lower prices will fall, and the more consumers will benefit. Thus, if a machine like an AI proves the better choice over human effort for any given task, then we should hope it is way, way better than we are at that task. If so, its output gets really cheap.

Of course, even if goods and services get much cheaper, people will still need a source of income in order to afford them. At this point, we have a tradeoff — automation makes things less expensive, but it also replaces human work. So how can the labor force win, in the balance? This brings us to the second critical feature of modern economies: bottlenecks.

Bottlenecks

Let’s return to farming and consider the price of corn. With the help of machines like combine harvesters, the price of corn at the farm gate is now only about 10 cents per pound of kernels. Yet cornflakes, corn syrup, and corn chips cost 20 to 40 times that (e.g., $4 for a one-pound bag of corn chips). This is not because the farm or some other business is charging giant markups over its corn costs. It is because there are many other steps and associated costs downstream of the farm, including transportation, storage, processing, packaging, and retail services. Each of these currently requires many labor tasks, many of which remain hard and unproductive. There is no “amazing machine” for stocking shelves, building the grocery store, or fixing a flat tire when a truck breaks down.

Tasks that cannot be automated end up receiving a large share of economic value. Here’s the critical and perhaps surprising result: the economy is largely a story of what we do badly, not what we do well. This follows the same logic whereby amazing machines make their output cheap. The related implication is that the worse we are at something, the more expensive it will be. The expensive parts of the economy tend to be the things we need to do but haven’t figured out how to improve. And these so-called bottlenecks are everywhere. Combine harvesters have made harvesting corn the wide part of the bottle. Getting corn from the farm to a corn chip in your hand? Therein lie many tasks that constitute the narrower part of the bottle. And that’s where the payments go.

Due to uneven productivity gains, we spend more on restaurant meals than computers. This phenomenon — that we end up spending most of our time and money on the things we are unable to automate — is known to economists as “Baumol’s cost disease.” It helps explain why agriculture and manufacturing have been declining as a share of US GDP, while services — education, health care, government, finance, insurance, transportation — take over larger shares of GDP. For example, in a restaurant, the productivity of chefs, prep cooks, servers, bartenders, and dishwashers has made at most modest advances over many decades. Today we spend several times more on restaurant meals than on our ubiquitous computers.

Despite AI tools for medical diagnosis, health care delivery remains labor-constrained. AI systems are already showing impressive capabilities in medical-imaging analysis, diagnosis, and administrative documentation. As these tools improve, the cost of clinical decision-making is likely to fall sharply. But cheaper and better medical assessment does not reduce the need for care. And the delivery of care itself remains constrained by human labor. Nurses, physicians, technicians, caregivers, and support staff have many roles to play. Hospitals must still be staffed. Treatments must still be administered; surgeries must be performed.²

According to this logic, successful automation does not eliminate the need for human work. Rather, it shifts the demand for labor, concentrating value and wages in the parts of the economy where human time remains the limiting factor.

Price dynamics and economic bottlenecks give us good reasons to be hopeful for labor. If labor continues to perform various bottleneck tasks, while machines get very good at everything else, we end up in a situation where labor does very well. In fact, let’s take it to the limit. Let’s say AI becomes so efficient that it can do tasks like writing computer code at close to zero cost compared with a human performing the same assignment. And let’s say AI advances similarly at a large share of other tasks. Then, what we pay AI and related machines, as a share of GDP, becomes very small indeed. And the share of total income that goes to workers? It goes up.

Three Caveats

The above arguments can help explain major historical transformations and show how advanced AI could be advantageous for workers. But there are several caveats to acknowledge, where the economic prognosis may become less rosy.

Marginally better AI

AI that only marginally outperforms humans could be a net negative for workers. Today’s plentiful food supply and diversified economy lean on machines that are incomparably more efficient than we are at many forms of agricultural work. But imagine if such tools had instead offered only modest productivity gains over human labor. Such “marginally better” machines could have led to lost jobs without meaningfully lowering food prices for consumers. Moreover, because the amount spent on the machines in such an economy would be high, they would take over a larger share of GDP, with labor getting paid a smaller share. This is exactly why we want AI to excel at what it takes over, not just provide a slight improvement.

That said, there are natural reasons to think that AI productivity gains will be more than slight. First, computers tend to be fast, producing a lot of calculations in a short period. Second, computers don’t cost very much per unit of time, primarily because electricity usage is less expensive than wages. Where AI is able to perform a task successfully, it seems likely to lower costs considerably.

Rapid job loss and a lack of alternative work

For jobs that are replaced, workers’ outcomes hinge on whether they can transition into new roles. A second — and potentially profound — concern is whether displaced workers will be able to find new employment. In flexible labor markets, workers can move toward bottlenecks, performing roles in which they may ultimately be better off. But the new job may be inferior to the old job for a given worker, especially if that person was especially skilled at the prior work. And job transitions themselves — which can involve financial strain, loss of identity, and household stress — are painful.

A major factor affecting whether workers can shift to alternative jobs is the speed of automation. Gradual change allows adjustment; rapid displacement can overwhelm workers and communities.

Abrupt automation can carry heavy social costs. History offers a vivid illustration. England’s Captain Swing riots, in 1830, followed the rapid adoption of mechanical threshing machines. These machines eliminated a key source of winter employment (grueling though it was) for farm laborers. As the machines spread, laborers rioted. They destroyed equipment, burned buildings, and threatened violence against parish authorities and landowners. This episode shows both the hardship workers can endure as a byproduct of automation and the broader social conflict that it can spark.

Alternative work options and political conditions are critical. Tellingly, areas of England with greater alternative employment opportunities saw few if any riots. Further, the unrest unfolded in a context of weak political rights, limited social support, and extreme poverty. Over the previous decades, England had been reallocating common land to large local landowners — so-called enclosure — creating a large class of landless farm workers who became dependent on wage work. Further, these workers could not vote. Thus deprived of property rights and political rights, they predictably revolted. The lesson, then, is not that automation inevitably leads to turmoil; it’s that having access to alternative opportunities and civic power both matter enormously to the social response. This insight informs how we might confront more extreme technology scenarios, as well as our third caveat: the prospect of full automation.

Full automation

What happens if AI exceeds human capacities at all tasks? Full automation of human work strikes me as still far off, because it requires AI that excels at all cognitive and all physical tasks, and it also requires that humans will want to be served by robots for virtually everything we care about. Count me as skeptical that this happens anytime soon. But let’s say that we do reach a future when humans can be fully replaced and no bottleneck tasks remain. What happens to incomes then?

Prices might drop close to zero, but workers might receive only a tiny fraction of value. If machines do everything, then those who own the machines will capture all this value. Products and services would become very cheap, but workers, outcompeted by machines in all tasks, would end up with a vanishingly small share of the economy’s income.³

Political systems would need to decide how to distribute machines’ vast output. The full automation scenario is so far beyond human experience that we should be careful in making any claims about it. But depriving the large majority of people in society of a means to support themselves, while a few people become incredibly wealthy, does not seem a stable situation. Returning to England in the 1830s: as the Swing riots progressed, the wealthy realized their own personal risk at the hands of workers with little left to lose. The social unrest intensified pressure for political change, contributing to the Great Reform Act of 1832, which began to extend voting rights and bring formal political power to England’s middle class. In short, when technological progress pushed workers into an untenable situation, people rose up, and political institutions shifted, working (if imperfectly) to accommodate these challenges. Similarly, political systems will be the essential arbiters in a world of full automation. While full automation would create vast output gains, it could create dystopia or utopia, economically, depending on how we share its fruits.

Labor will fare best if AI automates with extraordinary competency. These caveats matter. They remind us that technological progress is not automatically benign, that transitions can be painful, and that institutions shape outcomes. But it is also easy to let the most dramatic possibilities dominate our thinking. The future is unlikely to jump directly to dystopia or utopia. More plausibly, AI will advance unevenly — extraordinarily powerful in some domains, limited in others. And, in a world of profound yet uneven transformation, the fate of workers will again hinge on the same two questions that shaped agriculture and computing: how cheap automation makes what it does well, and where the narrow parts of the bottle remain. The promise of AI, then, is not that it preserves existing jobs but that, by collapsing prices in the tasks it masters, it pushes value to the tasks it does not master. So, if AI is going to beat us at a task, let’s hope it beats us by a mile. Then, ironically, labor may yet win.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Benjamin Jones is a Non-Resident Senior Fellow at IFP and the Gordon and Llura Gund Family Professor of Entrepreneurship and a Professor of Strategy at Northwestern University. An economist by training, Professor Jones studies the sources of economic growth in advanced economies, with an emphasis on innovation, entrepreneurship, and scientific progress. A former Rhodes Scholar, Professor Jones served in 2010-2011 as the Senior Economist for Macroeconomics for the White House Council of Economic Advisers and earlier served in the U.S. Department of the Treasury.

Last month, as three Chinese AI startups went public within days of each other, Hong Kong briefly became a scoreboard for emerging companies in the industry. On January 2, AI chip designer Shanghai Biren Technology listed in Hong Kong and raised $5.58 billion Hong Kong dollars ($717 million). About a week later, model developers Zhipu AI and MiniMax followed, raising HK$4.35 billion ($558 million) and HK$4.8 billion ($619 million), respectively.

Those listings matter less as a market spectacle than as a strategy signal, and the strategy differs noticeably from that of companies across the Pacific. US startups build around abundance: raise huge capital, buy time, push the frontier. OpenAI’s Stargate plan, for example, aims to invest $500 billion over four years in AI infrastructure. Meanwhile, Chinese startups adapt to different constraints: frontier training infrastructure is scarcer, so momentum comes from efficiency, targeted deployment, and market selection.

As AI moves from demos to production, the binding question shifts: what does it cost to deliver useful work reliably, and who will pay? Under different economic pressures, Chinese and US companies are opting for different go-to-market strategies.

The Capital Gap

US AI startups attract more private investment than those in China. The divergence starts with money. In 2024, US AI startups received approximately $109.1 billion in private investment, while Chinese AI startups received roughly $9.3 billion — a ratio of 12 to 1. Chinese AI funding fell 38% year-over-year in 2023. By 2025, foreign investors provided only 10% of Chinese tech startup funding, with pure-play AI companies seeing foreign participation below 12%.

Chinese government funding only partly makes up for a lack of private investment. One might think the Chinese government would fill the gap, and that is partially true. China’s Big Fund III has registered capital of 344 billion yuan ($47.5 billion), and local governments have introduced “computing vouchers” that subsidize up to 80% of cloud computing costs. But government capital operates differently. Georgetown’s Center for Security and Emerging Technology (CSET) finds Chinese guidance funds have a historical disbursement rate of roughly 50%, meaning that much of the committed capital remains idle. Government programs mitigate but do not eliminate the constraint.

Chinese tech giants are pursuing scale, but smaller players can carve out niches. A second reason to believe that lower capital will have limited effects is the idea that China’s AI development will simply concentrate in large corporations that can afford scale. With ByteDance investing an estimated $21 billion in AI infrastructure in 2025 and Alibaba announcing a 380 billion yuan ($53 billion) three-year plan in February 2025, it is clear that these giants aim to dominate.

But China’s market is vast and stratified: demand varies dramatically across regions and industries, from coastal manufacturing hubs to inland services, and from enterprise automation to consumer entertainment. This fragmentation creates defensible niches that no single player can fill. The independent Hong Kong listings of Zhipu and MiniMax reflect a market structure where scale alone does not guarantee capture — startups can win specific segments even as giants pursue the whole.

Who Pays, and What Are They Buying?

US startups can earn revenue by selling access to AI models. In addition to the capital gap, the question of who is paying for AI reveals sharper comparisons. In the US, startups sell “capability as product”: subscriptions with clear price points. ChatGPT Plus launched at $20 per month. GitHub Copilot starts at $10 per month. In high-salary markets, subscribers justify small monthly fees by arguing that these tools save time.

In China, consumers expect AI model access to be provided for free. Chinese consumer AI operates on a fundamentally different model. Following DeepSeek’s free-tier disruption, Baidu made Ernie 4.0 completely free for consumers in April 2025, abandoning its subscription experiment. ByteDance’s Doubao has been free since its launch, monetizing through API revenue and ecosystem integration rather than consumer subscriptions. The price anchor in China is not $7 or $5 — it is $0.

Chinese startups provide free AI access and monetize by other means. This reflects strategic logic, not charity. For Baidu and ByteDance, AI functions as a traffic platform, the next-generation search engine or super-app, where user acquisition matters more than subscription revenue. Monetization happens elsewhere: enterprise API calls, cloud service bundling, and advertising conversion. Lower per-capita income and the absence of established software subscription habits make this approach more viable than transplanting US-style pricing.

Chinese firms spend far less on software than US firms do. On an economy-wide basis, the World Intellectual Property Organization WIPO (drawing on S&P Global Market Intelligence) estimates 2024 software spending of $368.52 billion in the US compared with $61.8 billion in China (a ratio of roughly 6 to 1). Normalizing those totals by official employment figures, the implied spending is roughly $2,284 per employed person in the US compared with roughly $84 per employed person in China. These gaps help explain why many Chinese AI startups lean into institutional procurement tied to operational outcomes: the “product” often bundles deployment, customization, and accountability — not just model access.

Constraints Determine China’s Choices

For Chinese startups, constraints shape strategy in three major ways.

Chinese AI developers design models to minimize usage costs. First, developers aim for efficiency. DeepSeek-V3 is a Mixture-of-Experts model with 671 billion parameters, but only 37 billion activated per token. While MoE techniques are not unique to Chinese models, cost per token and cost per task are business variables; when capital is scarce and customers are price-sensitive, efficiency expands the addressable market.

Chinese AI startups often target international markets from inception. Second, constraints influence companies’ market positioning. MiniMax served more than 212 million users across 200-plus countries through AI companion and role-play apps like Talkie, with overseas markets contributing over 70% of revenue. This “born-global consumer” strategy works in entertainment categories where spending habits are established and value is experiential. While MiniMax’s scale is exceptional, other Chinese AI products like CapCut (for video editing) and Faceu (for augmented reality filters, among other features) have demonstrated similar overseas-first patterns, suggesting this approach is becoming a viable alternative to competing directly with US incumbents in productivity tools.

Chinese companies tailor hardware for inference, not training. Third, chip companies focus on deployment-oriented infrastructure. Biren’s IPO shows that hardware startups can succeed by optimizing for inference rather than promising to dominate frontier training.

Economic pressures are fueling faster industrial deployment in China. More broadly, capital and monetization constraints also increase pressure on Chinese startups to find valuable use cases faster. Evidence suggests this is happening. In manufacturing, 67% of Chinese industrial firms have deployed AI in production, compared with 34% of analogous US firms — roughly double the adoption rate. Deloitte notes that many US manufacturers remain in “pilot purgatory,” beginning scaled deployment only in 2026. In logistics, China’s JD Logistics has leveraged AI to offer 12-hour delivery in core cities, versus Amazon Prime’s 1 to 2 days. Cainiao’s AI-powered consolidation has cut cross-border delivery times by 50%.

This accelerated deployment is due to three dynamics. First, while US enterprises wait for frontier models, Chinese companies are more pragmatic, deploying open-source models with heavy fine-tuning to solve immediate problems. Second, lower compliance friction in China shortens procurement cycles. Third, Chinese AI companies more often sell end-to-end solutions rather than use-specific tools, incentivizing buyers to adopt deep workflow integrations. When revenue depends on delivering outcomes rather than access, speed to deployment becomes a survival metric.

What to Watch

The US leads in model capability, while China leads in widespread deployment. This difference demonstrates that there is more than one vision of success in the AI industry. If we interpret progress as meaning frontier model capability, then the US keeps its edge — Chinese models trail by approximately 7 months. If, however, progress means economy-wide deployment, then China may be ahead, with its constraint-driven strategies compounding faster in specific layers.

Economic signals are the best markers of progress for China’s AI industry. Over the next 6 to 12 months, the strongest signs of whether Chinese AI startups’ strategies are working will come from economics, rather than model benchmarks. Starting with inference pricing, if efficiency gains are real, cost per useful task should decline steadily, reflected in API discounting and enterprise contract terms. Looking at the mix of revenue streams in public filings will also be informative; shifts between consumer and enterprise segments reveal which go-to-market motion is actually scaling. Additionally, monitoring renewal and expansion rates will distinguish pilots from durable adoptions. And, finally, overseas revenue growth from Chinese consumer AI products will be the measure of whether the born-global path can sustain momentum through distribution alone.

The conclusion is not that one country has the only viable route to AI success. Startups in different regions face different buyers, constraints, and distribution channels, so they optimize accordingly. The next winners in the AI industry will be the companies that treat those realities as product requirements, and then scale what works.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Poe Zhao is a Beijing-based technology analyst who builds rigorous, bilingual explanations of China’s tech ecosystem for global readers. He founded Dailyio (Chinese) and Hello China Tech (English) to map how China’s AI, robotics, semiconductors and EVs scale in practice: from policy design and state capital to supply-chain execution and market adoption.

Cover image: Irina Shilnikova / iStock

This month, mainstream media have been warning consumers that electronic devices may get pricier because of rising demand for dynamic random access memory (DRAM), a key component. The surge in DRAM costs, estimated to have risen by 50% during the final quarter of 2025, can largely be traced back to a specific cause: the AI industry’s appetite for high-bandwidth memory (HBM). This demand has led memory-makers to shift production away from standard DRAM chips and toward HBM.

While its contribution is often overshadowed by those of processors it supports, HBM plays a vital role in training and running advanced AI systems, so much so that it now accounts for half the production cost of an AI chip. Companies’ determination to secure this lesser-known component proves its value. In December 2024, the US announced new export restrictions on the sale of HBM chips to China. In the month before the restrictions came into effect, Huawei and other Chinese companies reportedly stockpiled 7 million Samsung HBM chips, a haul likely worth over $1 billion.

The December 2024 controls specifically targeted HBM in order to slow China’s domestic AI chip-production efforts. Targeting HBM in this way is possible because it is manufactured separately from GPUs and then fused with them in a subsequent packaging step. Yet, while these controls are having an impact, they contain significant gaps. In the year since, Chinese companies have continued to acquire HBM directly via loopholes, and they have also purchased tooling needed to develop HBM domestically. If US policymakers are serious about slowing China’s AI chip production, they should close the gaps in HBM controls.

Why is high-bandwidth memory so important?

When people discuss AI chips, they usually talk about processing speed, measured in the number of floating point operations per second (FLOP/s). Processing speed is an important metric, as it tells us how quickly an AI chip can perform calculations when it has data to work with. But in practice, AI chips often sit idle, waiting for data to be fed to them.

Just as a factory’s output depends not only on how fast workers can assemble parts but also on how quickly parts reach the assembly line, so does memory bandwidth constrain overall AI chip performance. If AI chips are a factory, HBM is both the stockroom and the conveyor belt, storing and delivering parts to the workers fast enough to keep the assembly line moving.

HBM improves AI chip performance. In 1994, William Wulf and Sally McKee coined the term “memory wall” to describe problems arising from improvements in processor speeds far outpacing improvements in memory bandwidth. This observation proved prescient and drove efforts to increase memory bandwidth. In the mid-2000s, AMD began working on memory innovations that eventually led to HBM.

Traditional DRAM sits on separate chips connected to processors via the motherboard through relatively narrow channels. AMD’s key innovation was a series of procedures for stacking multiple DRAM dies on top of one another and using tiny, vertical electrical connections called through-silicon vias (TSVs). This configuration allows the layers to communicate. (A “die” is a piece of silicon circuitry that gets packaged alone or with other dies to create a chip.) These stacks are incorporated directly onto the same silicon wafer (an “interposer”) as the processor (for example, a GPU). At the bottom of the HBM stack sits a logic die that interfaces between the memory stack and the processor.

Because of the HBM stack’s position immediately beside the processor, the two can communicate across many more parallel channels than traditional memory allows. The result is dramatically higher data throughput while drawing less power.

HBM impacts AI progress. In December 2013, SK Hynix, which had worked with AMD to develop HBM, produced the world’s first HBM chip. In 2015, the company began high-volume production at its facility in Icheon, South Korea.

The first product to use HBM was the AMD Fiji GPU, in 2015: a consumer chip mainly used for video gaming. But HBM soon started appearing in data center GPUs and other AI chips, first with Nvidia’s Tesla P100, announced in April 2016. As AI models grew in size and complexity through the late 2010s and into the 2020s, HBM became effectively mandatory for high-performance AI accelerators. And as time went on, each successive generation of HBM provided AI chips with greater memory bandwidth and more memory capacity. In 2024, Nvidia announced its Blackwell architecture, which coupled each data center GPU with multiple HBM stacks.

Today, every major AI chip maker — including Huawei — uses HBM in its products. Those stuck with an older generation of HBM produce an inferior product, no matter how fast their processors are.

Mapping the global HBM industry

HBM production is highly lucrative and highly concentrated. Three companies control 97% of HBM wafer production today: SK Hynix and Samsung, of South Korea, and Micron, of Boise, Idaho. Because HBM is so vital to the AI industry, these companies now enjoy extraordinary market valuations. SK Hynix, the leading HBM producer, has seen its stock rise sevenfold since November 2022, when ChatGPT launched.

This concentration in HBM production reflects decades of global competition that left South Korean and US companies in command; Chinese players like ChangXin Memory Technologies (CXMT) are still racing to catch up.

China is a latecomer to HBM manufacturing. Just as cellular network technology has gone from 4G to 5G and so on, HBM technology progresses in generations. CXMT, China’s leading DRAM manufacturer, is currently manufacturing second-generation HBM chips — comparable to those first produced by South Korean companies in 2016. However, it is now reportedly skipping a generation, aiming to develop HBM roughly three to four years behind SK Hynix, Samsung, and Micron (see table below).

CXMT is attempting to close this gap by using large quantities of imported equipment, most notably stockpiled immersion deep ultraviolet (DUV) photolithography machines from the Dutch firm ASML. CXMT’s production also relies on equipment made by US firms like Applied Materials and Lam Research, as well as Japanese companies like Tokyo Electron.

Despite CXMT’s manufacturing advances, Huawei primarily uses more advanced HBM stockpiled from Samsung and, to a lesser extent, from SK Hynix. At its Connect conference in September 2025, Huawei announced plans for “proprietary” HBM, possibly to be fabricated primarily by CXMT or by Huawei itself with collaborators.

China is ramping up domestic HBM production. It remains to be seen how CXMT and other Chinese manufacturers may ramp up production over the coming years. Our best guess is that they will be able to produce approximately 7 million (primarily HBM3) dies in 2026. That is sufficient for about 600,000 AI chips of comparable performance to Nvidia’s H100 accelerator. This estimate assumes that each AI chip uses eight HBM stacks (as the Huawei Ascend 910C accelerator does) and a 70% yield (i.e., 7 in 10 packaged AI chips prove functional). However, these estimates are highly uncertain.

Even if these targets are met, China’s domestic production will likely fall far short of demand, leaving its AI ambitions fragile and highly susceptible to effective Western restrictions.

The gaps in current HBM controls

Because HBM is a critical enabling component for AI chips, and because AI capabilities have direct national security implications, in December 2024, the Bureau of Industry and Security (BIS), under the US Department of Commerce, rolled out export controls on HBM and related tooling. Its goal was to cut off China’s ability to both acquire and produce HBM more advanced than what the country can currently manufacture.

However, the rules and their application contain important gaps, detailed below.

Coverage for key equipment is incomplete. The December 2024 controls included a mechanism called Foreign Direct Product Rules (FDPRs) to target certain semiconductor manufacturing equipment (SME) — but with a key upgrade. Usually, FDPRs give BIS jurisdiction over foreign-produced items manufactured using US software or technology (e.g., designs, specifications, and know-how). The 2024 FDPR, however, gives BIS jurisdiction over any SME that “contains” a chip which was itself manufactured using US technology. These rules effectively capture the entire non-Chinese supply chain.

Importantly, though, the 2024 FDPR contains exemptions for firms headquartered in Japan, the Netherlands, and other nations with export-control regimes similar to those of the US. While the new rules reinforced restrictions of the most advanced immersion DUV lithography machines — crucial for much of HBM’s development to date — the informal agreements they rely on still allow Japanese and Dutch companies to sell older immersion DUV machines to China. Though these machines are less advanced, Chinese companies can use a technique called multi-patterning to make more advanced dies than they could otherwise, at the cost of higher defect rates and slower production.

The 2024 controls also contain gaps that allow China to scale up its semiconductor tooling industry. For example, the controls do not target equipment needed for hybrid bonding, a technique for reducing the height of the HBM stack to comply with industry standards, increase performance, and lower power consumption. Hybrid bonding will likely be critical for developing future generations of HBM.

An implementation gap allowed stockpiling. As previously described, the US rules’ one-month gap between announcement and implementation gave Chinese firms ample time to stockpile inventory. By then, the US government had been signaling that HBM controls were imminent for nine months. Chinese firms utilized this nine-month window aggressively. Huawei and Baidu stockpiled 6 million HBM stacks from Samsung — not counting the 7 million in the month between the rules’ announcement and their implementation. These stockpiles are enough to allow Huawei to manufacture about 1.6 million Ascend 910C chips (with memory bandwidth performance comparable to the Nvidia H100, introduced in 2022).

Stockpiling became rampant for chip-manufacturing equipment as well: according to one analysis, CXMT has likely acquired enough SME for HBM production through 2026 or 2027, after which it will encounter obstacles both in ramping up production volumes for current HBM generations and in developing more advanced HBM generations.

Equipment manufacturers received carve-outs. When introducing the 2024 rules, BIS also announced that it was adding 140 companies to the Entity List, which names foreign companies, organizations, and individuals that the US believes pose national security or foreign policy risks. US companies cannot do business with these entities without special licenses, which are difficult to obtain.

However, BIS diluted the final rules in response to diplomatic and economic pressures. For example, it placed CXMT on a narrower list that bars only the US Department of War from doing business with it, thanks to extensive Japanese lobbying on behalf of an equipment manufacturing company, Tokyo Electron, that relies on the Chinese market for 45% of its revenue. Such pressure from Japan illustrates how loopholes in the 2024 FPDR, described above, allow allied nations autonomy that may pose conflicts of interest when paired with US national security and chip export-control concerns.

To make matters even more complicated, Japanese regulation lacks sweeping abilities (which the US claims) to impose restrictions on specific companies that receive exports. As long as Japan can largely make its own rules under the 2024 FDPR, impeding CXMT’s progress will remain a challenge.

Current HBM controls incentivize extraction and smuggling. The December 2024 controls targeted the sale of raw HBM stacks, but they did not restrict HBM that had already been integrated into AI chips. In late 2024 and early 2025, several Taiwanese companies reportedly exploited this loophole by loosely attaching Samsung HBM to very simple processor chips. These chips were designed to appear compliant with export restrictions while allowing the HBM to be easily extracted once in China.

SemiAnalysis judges that this breach has now been contained, based on the offending firms’ revenues returning to normal levels. But the structural incentive to smuggle Western HBM stacks is high, because of their far superior performance and greater supply. Given that AI chip smuggling appears rife, it seems likely that bad actors will engage (or are already engaging) in similar diversions of HBM. Meanwhile, BIS’s enforcement, which remains heavily resource-constrained, cannot counter the strong incentives for these activities.

Tightening the regime

To make the controls more effective, policymakers can target each of the loopholes mentioned above.

Expand and modernize controls on key equipment. The US should pressure the Netherlands to ban the servicing of all existing ASML lithography tools in China, thus degrading their long-term viability. Depending on diplomatic momentum, the US could also work with the Dutch government to ban the sales of remaining immersion lithography tools to China, further narrowing the pipeline of chip-making tools available for HBM production.

Looking forward, controls should also target the next technical frontier: hybrid bonding. As the industry approaches the next few generations of HBM, the US should consider putting trade restrictions on companies that manufacture hybrid bonding systems, as well as overlay metrology (for measuring the alignment of one die layer to another) and plasma dicing (an etching process to perfectly separate dies from wafers), among other tools over which it has meaningful leverage.

Close the implementation gap. In addition, future controls should be issued with minimal advance notice to prevent the massive stockpiling China achieved ahead of the 2024 controls. While there is a trade-off between speed and allies’ buy-in, speed is essential for an effective export-control regime and should be prioritized over airtight allies’ approval. As evidence emerges in support of new controls, the US will need to act as quickly and with as little signaling as possible, while substantially closing the gap between action and diplomatic approval.

Stand firm on chip-maker sanctions. In its next round of controls, BIS should explicitly target the companies leading China’s domestic HBM efforts. CXMT, as the prime mover among those companies, should be added to the BIS Entity List so it cannot as easily acquire needed tools from foreign companies to advance HBM in quality and quantity.

Mindful of the diplomatic sensitivities of this decision, the US must convince Japan to either adapt its domestic export laws to mirror US restrictions or find alternative customers for Tokyo Electron. As a last resort, BIS could unilaterally revoke Japan’s exemptions from the relevant FDPRs. BIS should also target the broader ecosystem supporting Huawei by adding companies involved in chip-packaging to the Entity List and applying FDPRs to those making semiconductor equipment.

End component extraction and smuggling. BIS should issue “red flag” warnings to chipmakers, outsourced semiconductor assembly and test companies, and third-party vendors to watch for orders of advanced HBM memory packaged in ways that allow the memory to be easily removed from finished chips after export. It should also investigate whether Taiwanese or South Korean companies knowingly violated the controls by supplying Chinese companies with HBM stacks via lightly assembled products. Punishing the offending companies will send a strong signal to other would-be smugglers that semiconductor-related transactions with China are not worth the trouble.

Furthermore, Congress should authorize a whistleblower incentive program for individuals who expose export-violation schemes, such as HBM smuggling. This program should offer large, penalty-financed rewards, alongside strong protections such as anonymous reporting channels. Such a program has been proposed in the bipartisan and bicameral Stop Stealing Our Chips Act, first introduced in April 2025 by US Senators Mike Rounds (R-SD) and Mark Warner (D-VA).

Finally, to improve BIS’s enforcement capacity, Congress should grant the President’s budget request of $303 million for the agency, a 60% increase over its current budget.

Conclusion

Export controls must track China’s real capabilities. Ultimately, the US must calibrate its controls to the best chips and tools China can realistically make at large quantities domestically, not what America can achieve at the frontier. Otherwise, the controls risk allowing exports that ease China’s key domestic bottlenecks. By conducting regular audits of China’s capabilities, the BIS’s internal evaluators can ensure that export controls remain flexible enough to keep US firms relevant in the global market, while strict enough to prevent China from acquiring HBM capabilities that far exceed its domestic production potential.

Tooling controls must be less vulnerable to diplomacy. The efficacy of future HBM controls is complicated by the current diplomatic climate. In November 2025, for example, BIS suspended its “Affiliates Rule,” which would have extended Entity List restrictions to majority-owned subsidiaries of listed firms. This suspension, which lasts one year as part of broader US-China trade talks, will enable listed firms to continue accessing controlled technologies through overseas subsidiaries and affiliates. The suspension came after reports, in May 2025, that BIS was considering adding CXMT to the Entity List, a move that now seems to have been shelved for similar diplomatic reasons.

The most recent example of the political complications of new chip restrictions is the Trump administration’s decision to allow China to buy Nvidia H200 accelerators and comparable chips, signalling a more dovish approach to controls on Chinese AI chip-making. The H200 offers considerably higher memory bandwidth than current Chinese alternatives, allowing Chinese companies to enjoy better chip performance without having to import HBM stacks for domestic AI chips.

However, the H200 decision was motivated in part by a desire to counter Huawei’s AI chip efforts. The strategic rationale is that Chinese companies with access to US chips won’t want to buy inferior Chinese competitors. Strong HBM controls are essential for buttressing that strategy. Despite the aforementioned issues with the HBM controls, the technical gap remains a significant barrier for China’s AI industry. While Huawei and CXMT are aggressively pursuing domestic HBM development, limited domestic Chinese production, for now, will curtail the amount of advanced AI chips Huawei can produce, and thus limit China’s ability to surpass the newly available H200s.

Every loophole grants China another opportunity to close the technological gap. This gap may shrink faster than anticipated unless the US fully commits to tightening its regime. It may still be possible to contain China’s chip-making ambitions, but, to do so, the US must first close the exits.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Erich Grunewald is a researcher on the compute policy team at the Institute for AI Policy and Strategy (IAPS), where he focuses on export controls and data center security. He previously worked as a software engineer and earned a BSc in computer engineering and an MSc in interaction design from Chalmers University of Technology. He edits The Substrate, a blog about AI hardware.

Raghav Akula was a 2025 AI Policy Fellow at the Institute for AI Policy and Strategy (IAPS), where he focused on compute governance and U.S.-China strategy. He is also a student at Georgetown University’s School of Foreign Service with prior experience at the Office of the Director of National Intelligence (ODNI), MITRE, and an AI startup.

Last November, seven families filed lawsuits against frontier AI developers, accusing their chatbots of inducing psychosis and encouraging suicide. These cases — some of the earliest tests of companies’ legal liability for AI-related harms — raise questions about how to reduce risks while ensuring accountability and compensation, should those risks materialize.

One emerging proposal takes inspiration from an existing method for governing dangerous systems without relying on goodwill: liability insurance. Going beyond simply compensating for accidents, liability insurance also encourages safer behavior, by conditioning coverage on inspections and compliance with defined standards, and by pricing premiums in proportion to risk (as with liability policies covering boilers, buildings, and cars). In principle, the same market-based logic could be applied to frontier AI.

However, a major complication is the diverse range of hazards that AI presents. Conventional insurance systems may be sufficient to cover harms like copyright infringement, but future AI systems could also cause much more extreme harm. Imagine if an AI orchestrated a cyberattack that resulted in severe damage to the power grid, or breached security systems to steal sensitive information and install ransomware.

The market currently cannot provide liability insurance for extreme AI catastrophes. This is for two reasons. First, the process of underwriting, which involves assessing risk and setting insurance prices, requires a stable historical record of damages, which does not exist for a fast-moving, novel technology like AI. Second, the potential scale of harm caused in an AI catastrophe could exceed the capital of any single carrier and strain even the reinsurance sector.

In other words, conventional insurance covers frequent, largely independent events that result in a limited amount of harm. AI risks, meanwhile, are long-tailed. While most deployments will proceed without incident, rare failures could snowball into damages beyond the capacity of the insurance market to absorb.

Catastrophe bonds can fill the gap left by conventional insurance. As an alternative market-driven solution to liability insurance, we propose AI catastrophe bonds: a method of insurance tailored to cover a specific set of large-scale AI disasters, inspired by how the market insures against natural disasters. As well as offering coverage, our proposal would be structured to encourage higher safety standards, reducing the likelihood of all AI disasters.

Catastrophe Bonds: A Blueprint for Insuring AI

A proven capital-markets model for tail risk already exists. AI may be a novel technology, but it is not the first phenomenon to present the challenge of rare, extremely harmful events. From hurricanes in Florida to earthquakes in Japan, natural disasters offer a precedent for dealing with such tail risks. In these cases, the insurance industry packages liabilities in the form of financial assets called insurance-linked securities (ILS). These can be sold to global capital markets, transferring specific insurance risks to the deepest pool of liquidity in the world. Investors receive attractive returns in exchange for bearing potential losses from designated catastrophic events.

The most notable example of ILS are catastrophe (cat) bonds. Typically covering natural disasters and cyberattacks, cat bonds are designed to offload extreme risks that conventional insurers struggle to hold. A reinsurer sponsors a bond that pays a high yield but automatically absorbs losses in the event of a predefined disaster. In effect, capital markets become the backstop for extreme events.

Natural buyers of cat bonds include hedge funds, ILS funds, and other institutional investors: they have both the sophistication to price complex, low-probability risks and the appetite for asymmetrical, nonlinear payoffs. In return for accepting the risk of total loss, they receive outsized yields.

Here’s how cat bond money flows among stakeholders:

Taking this mechanism as a blueprint for AI, frontier developers could similarly sponsor “AI catastrophe bonds,” transferring the tail risk to a pool of capital-market investors, who would accept it in exchange for high yield.

Catastrophe bonds can benefit AI developers. Although AI developers are not currently automatically liable for all AI-related harms, they could nonetheless stand to benefit from issuing cat bonds. Several developers are already facing lawsuits, such as those described earlier, alleging that their products have caused mental illness and wrongful death. Given the long tail of potential dangers, companies may prefer to obtain protection from major losses, even if the harms for which they may bear liability have not yet been extensively tested in court.

Regulations could make coverage mandatory. Beyond AI developers’ own motivations, regulators could also mandate that frontier developers maintain minimum catastrophe bond coverage as a licensing condition, similar to liability requirements for nuclear facilities or offshore drilling.

In technical terms, an AI developer would issue a cat bond through a type of legal entity called a special purpose vehicle (SPV). When investors purchase the bond, their funds are held as collateral inside the SPV (typically in safe, liquid assets). In normal times, the developer pays investors a “coupon,” which is economically analogous to an insurance premium. If a defined AI “catastrophe” trigger occurs (more on that below), some or all of the collateral is released to fund payouts, and investors absorb the loss.

Investors could buy AI cat bonds as part of a hedged strategy. It is worth noting one important difference between AI risks and the types of risks traditionally covered with cat bonds: correlation with market performance. Natural disasters are largely uncorrelated with the financial markets, making the cat bonds covering them more appealing to investors. On the other hand, an AI-related event severe enough to trigger a payout would probably prompt a sharp repricing of AI equities.

This effect means any losses would likely occur at a particularly bad time for investors, which might reduce appetite for AI cat bonds specifically. Yet it also opens the door to hedged strategies, such as earning interest from AI cat bonds while betting against tech stocks. In this sense, AI cat bonds would make the tail risk of AI tradeable for the first time.

A Market-Driven Safety Mechanism

Cat bonds can incentivize safety, not just transfer risk. As well as helping to cover damage from catastrophes, it is equally if not more important that AI cat bonds reduce the likelihood of such events happening in the first place. As with regular insurance, the bonds therefore need to financially incentivize rigorous safety standards. This could be done by setting each developer’s coupon at a base rate plus a floating surcharge that reflects that developer’s safety practices.

A Catastrophic Risk Index would tie pricing to standardized safety assessments. For this to work, there would need to be a standardized, independent assessment of AI developers’ safety posture and operational controls, which we’ll call a Catastrophic Risk Index (CRI). Like credit ratings in debt markets, the CRI would translate safety practices into a transparent cost of capital. Safer labs pay less and riskier labs pay more, directly aligning incentives to reduce risk.

Consider this visualization of AI cat bonds’ potential structure:

The building blocks for a credible CRI are already emerging. Constructing a credible CRI is not a trivial undertaking, but the necessary infrastructure is already emerging. Dedicated AI safety organizations — such as METR, Apollo Research, and the UK AI Safety Institute — have developed evaluation frameworks for dangerous capabilities and alignment robustness. Meanwhile, bodies like the AI Verification & Evaluation Research Institute (AVERI) are being set up specifically to advance frontier AI auditing. An industry consortium or standards authority could consolidate these inputs into a unified, transparent index.

This would bring yet another safety benefit: for AI cat bonds to be investable, frontier developers would need to publicly disclose technical and operational risk metrics and submit to third-party audits. This transparency is a feature, not a drawback: it would provide the inputs necessary for the CRI, allowing capital markets to form independent price views rather than relying on guesswork.

The market for AI cat bonds could start small and develop alongside safety evaluations. Initially, while CRI methodologies mature, the first AI cat bonds would be modest and bespoke. But, as the index gains credibility, and as trigger mechanisms for payouts prove robust, the market could scale rapidly. The most convincing demonstrations of trigger robustness would be actual incidents that test the legal system’s capacity to assign responsibility and cat bonds’ ability to pay out fairly and efficiently. Yet, in the absence of such incidents, market confidence could still develop through rigorous third-party oversight, independent auditing of the trigger criteria, and broader institutional trust in leading AI developers.

As a knock-on effect, market confidence could catalyze secondary insurance and reinsurance markets. Traditional reinsurers that are currently unable to warehouse AI tail risk would gain a hedging and price-discovery tool through the AI cat bonds. Simultaneously, these bonds would offer investors a new form of insurance-linked security and a direct way to express a view on AI risk. The CRI would provide a basis for relative-value analysis: investors could compare risk-adjusted yields across companies, rewarding safer operators with lower premiums.

Trigger Conditions

In practice, AI cat bonds will require precise specification of when payouts should happen. Existing cat bonds have a range of “trigger mechanisms,” each with distinct advantages and disadvantages.

Payout triggers could be court decisions, measurable events, or industry-wide claim thresholds. One tradeoff is between the speed of payouts and their accuracy. For instance, a trigger mechanism based on court-awarded damages is likely to be accurate but slow, leaving victims uncompensated while investors endure prolonged uncertainty. On the other hand, parametric triggers, which are based on objectively measurable events (for example, an AI autonomously gaining access to resources above a certain threshold), would enable faster payouts, but present the challenge of specifying which measurable events reliably indicate catastrophe.

A third option is an “index-based” trigger mechanism, which would release payouts when total AI-attributed losses surpass a specified level across the whole industry. This avoids company-specific attribution disputes, but it could weaken safety incentives by sharing the costs of failures across developers, potentially diluting the consequences of inadequate safety standards for any individual developer. CRI-linked premiums as described above could help to mitigate this effect, however.

Market experimentation will determine which triggers work best. The most practical near-term design may combine elements of these trigger mechanisms to balance their advantages and disadvantages. For example, a parametric trigger might place funds in escrow upon detection of specified technical failures, with final disbursement happening after court confirmation of liability. Early AI cat bonds will likely experiment with multiple structures, and market selection will determine which designs are most effective.

How much could AI cat bonds cover?

AI cat bonds should give investors high returns due to uncertainty about loss events. To attract investors, the entities issuing cat bonds typically pay an annual coupon several times larger than the annual expected loss as a percentage of investors’ funds. The coupon is calculated as a base rate of the annual expected loss, plus that expected loss multiplied by a number called a “risk multiple.” Historical catastrophe bond data from Artemis suggests risk multiples of roughly 2–5, with higher multiples for novel or poorly modelled risks.

For AI catastrophes, the true probability of a covered loss event is highly uncertain. We estimate an annual expected loss of 2% of the funds in the SPV and a risk multiple of 4–6. The developer’s annual payment would then be 2% + (4–6)*2% = 10%–14% of the funds invested in the SPV. This pricing would be broadly consistent with historical ILS treatment of novel, high-uncertainty risks.

The AI industry could plausibly support an initial collateral of up to $500 million. From the above chart, we can estimate the initial market size. Frontier AI labs already spend hundreds of millions annually on safety research and compliance, so it is not implausible that they might assume an insurance premium of ~$10 million. If five major labs were to participate (for instance, Google DeepMind, OpenAI, Anthropic, Meta, and xAI), aggregate annual premiums would total approximately $50 million. If this amount represents 10%–14% of the funds invested into the SPV, this would support a collateral of $350 million to $500 million available to draw from in the event of a covered catastrophe. This estimate is comparable to mid-sized natural-catastrophe bonds, but it represents a floor; regulatory mandates, broader industry participation, and growing investor confidence could, within years, expand the market to $3 billion and $5 billion.

If investors showed a lack of demand, this would itself be informative: bonds failing to sell at plausible prices would signal that the underlying risk of an AI catastrophe may be higher than developers or regulators have assumed.

Conclusion

In summary, we propose AI catastrophe bonds: a way for frontier AI developers to buy protection against extreme, low-probability harms without engaging the insurance industry. Frontier AI risks are largely uninsurable through conventional channels: insurers lack the historical data to price policies and face risk profiles that don’t fit their risk appetite. Capital markets investors (like ILS funds and hedge funds), however, are well-suited to taking the other side of such bets, much as they already do for earthquake and hurricane risks. Meanwhile, several AI safety organizations are already developing credible risk-evaluation methods, which are necessary to continually inform and update the premiums that AI companies pay.

AI catastrophe bonds would benefit developers, investors, and society. This variable pricing is the key: each developer’s premium rises or falls with an independent index based on audited safety and governance metrics of that developer. Such a framework would create strong financial incentives to improve safety standards, reducing the likelihood not only of catastrophes covered by the bonds but also of worst-case, extinction-level scenarios. This is a win-win-win: labs get coverage, investors get a handsome risk premium, and society gets a market-driven safety mechanism.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Daniel is the co-founder and CEO of Exona Lab, a startup focused on AI risk quantification and exposure management. Before that, he was a Quantitative Analyst at Bank of America in the counterparty credit risk team focusing on understanding systemic risks in the financial system. He graduated from Imperial College London with a degree in Bioengineering.

Gabriel is an Associate Professor at Touro University Law Center and a Non-Resident Senior Fellow at the Institute for Law & AI. He also serves on the board of Principles of Intelligence. His research focuses on the role of liability in mitigating catastrophic AI risk, and he regularly consults with legislators and other policymakers on AI policy matters. Before joining the Touro faculty, Professor Weil held several positions focused on climate change policy, including work for the Climate Leadership Council and the White House Council on Environmental Quality. Professor Weil holds a J.D., cum laude from Georgetown University Law Center, an LL.M. in environmental law, summa cum laude, from Pace University Elizabeth Haub School of Law, and a B.A. in political science, physics, and integrated science from Northwestern University.

Last week, the US government gave chip-maker Nvidia the green light to sell its H200 graphics processing units (GPUs) to approved buyers in China. These GPUs were previously subject to export controls preventing their sale to China. Following Nvidia’s record-breaking $5 trillion valuation, in October, this approval squares neatly with the views of many US policymakers who argue for an AI strategy focused on exporting US technology at scale. Among them is Sriram Krishnan, senior White House policy advisor on AI, who has stated the economic motivations bluntly: “Winning the AI race = market share.” Krishnan’s approach contrasts starkly with that of the Biden administration, which aimed to defend American AI leadership and protect against national security threats through increasingly stringent restrictions on exports of US AI hardware.

The White House’s July 2025 AI Action Plan, which lays out its export-focused strategy in depth, argues that selling the “full AI technology stack — hardware, models, software, applications, and standards” is the key to preventing other countries from adopting rivals’ solutions instead. The US could thus entrench its position as the leading provider of AI, the rationale goes, and secure long-term influence over not only AI hardware but also the AI models and applications used around the world.

In practice, much of the focus has been on exports of AI hardware, given strong global demand for Nvidia GPUs. However, a strategy that emphasizes AI chip exports as the path to US leadership will likely fail to translate into a lasting competitive advantage across the AI stack. It is not safe to assume that increased market share at one layer of the stack automatically promotes market share in other layers — if anything, the opposite sometimes proves true. Diffusion of America’s most advanced AI technology to the rest of the world could also weaken US national security if poorly managed. Meanwhile, there are smarter, more structured approaches to offering other countries access to American AI. Renting chips in US-based data centers, for example, could bring the US many of the same — or even more — economic benefits, while mitigating risks effectively.

Exporting Chips Weakens Other Parts of the US AI Ecosystem

Ramping up chip exports would increase Nvidia’s market share in China, but, as far as the wider US AI ecosystem is concerned, the advantages might end there. In fact, selling chips will likely increase competition for US companies at the model and application layers, because there is nothing to stop customers with US chips from training or running non-US models on them. Selling more hardware may therefore mean that AI developers based in China and elsewhere have more resources to develop competing models. Meanwhile, US AI companies have faced shortages of AI chips and are competing for supplies of these with companies outside the US. Therefore, increased AI hardware exports could also weaken the ability of US companies to accumulate enough AI chips to train models at the frontier of technology.

US chips are already being used to train and run the vast majority of non-US models. Last year, out of 103 AI models created by Chinese AI developers, 100 were trained on US hardware. Nvidia is the largest supplier of AI chips to China, where they are primarily used to train and run Chinese models, demonstrating how access to American hardware is accelerating the growth of China’s AI industry. Additionally, even exporting chips to other countries outside China might indirectly give Chinese AI developers more resources to train their models.

This is in part due to chip smuggling, enabled by poor enforcement of export controls; reports suggest that American AI chips smuggled to China in 2024 numbered in the tens to hundreds of thousands. Yet, even if GPUs stay in the countries that imported them, those countries may end up renting them out — especially if they do not have a well-developed strategy for how to use them domestically — and the largest non-American customers would likely be Chinese AI developers. This is already happening on a large scale in Malaysia and other countries in Southeast Asia, where Chinese developers such as ByteDance and Alibaba are accessing large volumes of US chips.

More availability of chips outside the US currently favors Chinese models, not US ones. Rather than putting all developers on an even footing, greater availability of AI chips might actually promote the use of non-US models above those from the US. This is because many enterprise or government users may prefer to run open-source models — which give them better privacy and customizability — and the majority of the most capable open-source models are Chinese. An early indication of this possibility came in September, when the open-source model family Qwen, developed by Alibaba, overtook Meta’s Llama models as the most popular open-source model in the world. Many users of open-source Chinese models are Western companies running them on American chips.

The drive to export is based on learning the wrong lessons from past China-US competition. Given that the widespread proliferation of US chips does not necessarily create broader dependence on American models and applications, why are many policymakers convinced that it does? One explanation may be that policy-makers are fighting the last war and trying to learn lessons from previous Chinese successes in technology competition, such as Huawei’s exports of 5G infrastructure. Michael Kratsios, who directs the White House Office for Science and Technology Policy, has drawn this analogy, noting, in an interview with the Center for Strategic and International Studies, that other countries “didn’t want to buy [Western alternatives to Huawei telecommunications systems] because the U.S. just was not able to create the environment and the packages necessary to export it out.” Advocates of an export-led strategy argue that the US cannot allow the same to happen with AI.

Yet this comparison does not stand up to scrutiny. In 5G, hardware and software are tightly integrated; if a country has Huawei 5G equipment, it must use Huawei 5G software as well. AI chips, on the other hand, are designed as general-purpose hardware for training and running a wide range of models. Once the GPUs are sold, US companies have no control over what customers use them for and cannot ensure any continuing returns at other layers of the AI technology stack.

Pro-export policymakers have acknowledged these issues. In fact, Dean Ball, primary staff author of Executive Order 14320, “Promoting the Export of the American AI Technology Stack,” has recognized this problem himself. Writing last month about what the EO was trying to achieve, Ball acknowledged: “We could end up constructing data centers abroad—and even using taxpayer dollars to subsidize that construction through development finance loans—only to find that the infrastructure is being used to run models from China or elsewhere.”

To avoid this outcome, he suggests that American AI model developers could, with government support, work with US hardware providers and data center operators to build infrastructure and run AI services in other countries. Ball points out that some developers are already starting to pursue a strategy of this kind. “OpenAI for Countries,” for example, seeks to build data centers abroad and to offer versions of ChatGPT tailored to local needs, while protecting democratic principles and continuing to improve safety standards.

By providing comprehensive solutions and ongoing support, partnerships like this could promote US market share in AI models over the longer term, helping to address concerns around security and lasting economic returns. But in practice, most sales currently being made are not within this framework. The deals announced so far have predominantly been straightforward exports of hardware, often without any publicly announced conditions to address the issues mentioned above.

China Does Not Have Enough AI Chips to Seize Market Share from the US

Even if selling US chips would not boost the use of American models around the world, some may argue that the US must export hardware anyway, to defend market share at the hardware layer. David Sacks, who chairs the President’s Council of Advisors on Science and Technology, recently voiced this anxiety, posting on X: “China is exporting Huawei chips + DeepSeek models to the Global South. If we don’t make it just as easy to export the American AI stack, we will forfeit this technology race in large parts of the world.” Yet current evidence suggests that Sacks’s concern is unfounded in the near term.

China is not producing enough AI chips to meet its domestic demand, let alone for export. Production of Huawei’s Ascend AI chips — estimated at 805,000 units this year — still falls far short of China’s domestic demand, which continues to be met in large part by Nvidia. Huawei is not exporting AI chips yet, and it is unlikely to do so before it can satisfy hardware needs within China. Industry analysts forecast that unless Huawei, China’s largest AI chip manufacturer, can secure new imports of High Bandwidth Memory (HBM), which is currently subject to US export controls, its production of AI chips will actually fall next year.

Chinese AI chips will likely not match the performance of American chips for several years. Even according to Huawei’s own announced roadmap, it will not produce a chip that can compete with the Nvidia H200 on performance until Q4 2027 at the earliest. In addition, Chinese AI chips suffer from issues with lower reliability, primarily due to a software ecosystem that is less mature and well-developed than Nvidia’s Compute Unified Device Architecture (CUDA). This makes it significantly more expensive and time-consuming to train AI models using Chinese chips relative to Nvidia’s.

The Impacts of Exporting AI Chips and Open-Weight Models on National Security Cannot Be Ignored

Exports of AI chips are unlikely to guarantee long-term US dominance across all parts of the AI industry. They also carry real national security risks that need to be considered alongside the economic impacts.

Exporting chips could enable adversaries to train models that pose a threat to the US. AI models are generally a dual-use technology, with civilian uses as well as military applications (including command and control, autonomous weapons systems, and cyberattacks). Exporting GPUs en masse could significantly expand the pool of actors with the necessary resources to train and run powerful, hazardous models — the kind top frontier labs typically keep walled off until after they have established proper guardrails. As highlighted above, the US will have no control over how its chips are used once they are exported. Selling them in large numbers, particularly beyond close allies, could therefore pose a serious national security threat to the US.

This risk is not unknown to pro-export policy-makers. Indeed, in April this year, it was part of the motivation for introducing stricter controls on exports of Nvidia’s H20 chips to China. In relation to that policy, Krishnan noted: “When it comes to inside China, I do think there is still bipartisan and broad concern about what can happen to these GPUs once they are physically inside.” Yet in the months since then, this concern seems to have been pushed aside, as the White House first reversed the restrictions on the H20, and then announced it would allow sales of an even more powerful chip — Nvidia’s H200 — to China as well. Moreover, with Nvidia’s advanced Blackwell GPUs being exported to other countries, there seems to be little acknowledgement that actors outside China could also use the chips in ways that threaten the US.

Adversaries with access to the weights of US models could bypass guardrails. Supporters of an export-focused AI policy might argue that this is why the US needs to export the full stack; by providing access to US models alongside the hardware, America could ensure that the default models in use reflect Western values and have guardrails in place to prevent their deployment in foreign military applications, for example. Yet, unless this is done under a carefully considered framework in which developers retain ongoing oversight, as Ball suggests, simply offering American models would do little to mitigate the risk; with sufficient hardware and access to a model’s weights, customers could retrain it to circumvent any intended limitations on its use. Today, this risk is largely theoretical, since most of the leading open-source models come from China. But if America were to regain the lead in developing open-source models and then export these to the world, providing access to a sophisticated model, even one with guardrails, could expedite adversaries’ activities, since retraining the existing model might be easier than trying to train a powerful new model from scratch.

Rather than maximizing exports at all layers of the AI technology stack, the US should develop a more nuanced strategy that balances economic goals with security concerns.

We Can Balance Economic and Security Goals by Renting America’s AI Technology Instead of Selling It

Fortunately, the US need not choose between the goals of capturing the benefits of global demand for American AI chips, retaining leadership in other parts of the AI stack, and protecting US national security. Multiple alternatives to one-time GPU sales achieve a more attractive balance between these priorities. Ball’s proposed framework could work well for some countries, but there is no one-size-fits-all solution. For example, in cases where nations want more freedom around which models they use, another approach would be to rent access to US-based hardware. The degree of model control and customization would depend on a country’s objectives for the rental and their alignment with US safety and security standards.

Renting chips could be more economically beneficial than selling them. Instead of selling large quantities of AI hardware, the US could focus on providing greater cloud access to chips hosted in data centers within the US or closely allied countries, as Janet Egan and Lennart Heim have proposed. For American companies, this strategy would have many of the same economic benefits as selling chips. In fact, it could be even more advantageous, providing a continuous revenue stream over the long term rather than one-off sales.

US cloud providers could vet users of AI chips. Unlike selling chips, however, renting them would grant the US meaningful oversight and control of how they are used. This makes it a far better proposal in terms of national security. At the user level, providers could implement know-your-customer requirements to prevent their compute being accessed by adversaries. This would also allow the US government to impose restrictions on which countries could access the GPUs by, for example, preventing China from renting them. This stands in contrast with exporting hardware, which gives companies no control over, or even knowledge of, where their chips end up or by whom they’re used, as chips can easily be resold or smuggled.

US cloud providers could detect and prevent prohibited activities. As well as vetting customers, US-based cloud providers could also monitor how chips in their data centers are being used, identifying and preventing prohibited activities (such as large training runs that could produce models at the frontier of capabilities, which could pose heightened risks and require extra guardrails). Additionally, customers could use American models without getting access to their weights, meaning they would not be able to retrain them to remove safeguards.

Hardware-only sales undermine the security advantages of deals that offer more control. Whether a country would opt to rent US-based chips or to have its own infrastructure built locally through an “OpenAI for Countries”–style partnership would depend on its priorities. Those seeking more freedom to use open-source models, for example, would prefer the former, while those wishing to avoid dependence on data centers abroad would choose the latter. But the security advantages of either solution will be undermined if the US continues to pursue large volumes of hardware-only sales as well.

If the drive to export as much as possible leads to large chip sales with no strings attached, similar to those made in recent months, it will amplify the national security risks of AI while failing to guarantee long-term predominance of American technology or lasting advantages for the US AI industry. America can make much better use of its technological lead, either by exporting in a carefully structured way or by renting access to US-based hardware. Reaping the economic benefits of AI does not require compromising US security.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Laura is a staff writer for AI Frontiers. She has worked in science communication for over six years, both in press offices and at magazines. She studied physics at Imperial College London and trained in science communication at the European Southern Observatory.

Cover: Douglas Rissing / iStock

Earlier this year, Anthropic CEO Dario Amodei warned that powerful AI could render upwards of 50% of white-collar jobs redundant, with the impact concentrated on entry-level jobs. If these predictions hold true, it could imply a long-term crisis of skill acquisition. Without the training ground of a first job, young workers could be denied the experiences and networks necessary to enter white-collar work. Their career trajectories could be severed before they begin.

AI is likely to disrupt more than the professional trajectories of individuals. The threat to career development mirrors a broader geoeconomic threat AI poses to developing countries: just as young workers need entry-level roles to climb into more senior roles, developing nations need viable “entry-level” industries to develop their human capital and ascend the global economic development ladder.

The Development Ladder

Many economies have followed a similar path for development over the past several decades. The most reproducible strategy has traced a familiar sequence: moving from low-skill agrarian production, to building a globally competitive manufacturing base, and eventually to exporting higher-value services and technology.

Such a progression is often described as a development ladder — a series of rungs that countries climb as they accumulate the capital and capabilities to compete in the global marketplace. This ladder is at risk from the coming wave of AI-driven automation.

South Korea is a canonical example of a successful development ladder. In 1953, its GDP per capita was $67 — among the lowest in the world, with little infrastructure and no natural resources. To bring the country out of poverty, President Park Chung-hee’s government made a bet on competitive manufacturing. Korea would compete globally, starting with the simplest products.

Over the next 50 years, Korea built entirely new industries from scratch: steel mills without any iron, petroleum refineries despite importing all oil. Textile manufacturing in the 1960s led to heavy industry in the 1970s, which spurred the development of an electronics industry in the 1980s. By 2020, South Korea’s GDP per capita reached $33,000: a nearly 500-fold increase in just 70 years.

Though Korea had other critically necessary benefits, such as strong pro-capitalist economic institutions and strategic American support, its ascent depended on a long window of successful export growth. At the heart of this success was labor arbitrage: Korean workers could produce goods at a fraction of the labor cost in developed economies, making Korean exports highly competitive.

Labor arbitrage has been the key enabler of export-driven development. There are dozens of determinants for the success of countries leveraging the export-led development ladder. Crucial requirements have historically included strong political and economic institutions, effective education systems, and access to major waterways. Developing countries benefit in comparison to their developed counterparts from reduced land costs, minimal regulatory burdens, and occasionally, generous tax incentives. However, perhaps their strongest driver has been wage competitiveness.

Export-driven development ladders are optimized around abundant and relatively inexpensive human labor. A garment worker in a developing country earning $3 a day, competing against an American earning $150, creates a 50:1 wage differential that makes this comparative advantage inevitable. This holds true even after accounting for shipping costs, quality variance, and supply chain complexities.

In many ways, this system is mutually beneficial for both producers and consumers. Consumers in wealthy countries receive access to cheaper goods. Developing countries gain capital investment and access to global markets, but also something far more valuable: systematic capability-building.

Labor arbitrage unlocks subsequent rungs on the development ladder. Through what development economists call “learning-by-doing” spillovers, foreign investment also inadvertently transfers technology and know-how that can eventually transform developing economies. The low-paid workers assembling electronics learn precision manufacturing, supervisors learn quality control systems, and so on. These workers eventually leave — starting businesses, joining competitors, or training others — and over time, dispersing skills throughout the economy.

The Evolution of Export-Driven Development

Exemplified by East Asian states such as South Korea, Japan, China, and Taiwan, countries prioritizing export-driven manufacturing strategies have lifted hundreds of millions of people out of poverty since the 1960s. Large-scale manufacturing has succeeded by transferring capital, technology, and learning-by-doing to massive, previously unskilled labor forces over decades.

Today, countries such as India, Vietnam, and Bangladesh are betting on growing manufacturing industries to continue their development. However, they are also facing headwinds: the past two decades have displayed persistent trends of premature deindustrialization, in which modern countries are running out of industrialization opportunities sooner and at much lower levels of income compared to earlier developing countries.

Digital services have been emerging as an alternative to manufacturing. More recently, attention has shifted in many developing nations towards exporting digital services. Contemporary developing countries, especially those in Sub-Saharan Africa, have explicitly positioned themselves around fintech, business process outsourcing (BPO), and information technology (IT) rather than attempting to replicate East Asia’s manufacturing-led model.

This is seemingly a rational response for the 21st century: why invest decades in building labor-intensive manufacturing when you can leverage educated populations, mobile connectivity, and English proficiency? By working remotely for companies in the developed world, locals in developing economies earn 2-4 times higher wages than manufacturing, and in better working conditions. Through this approach, many developing countries hope to leapfrog directly from agriculture to knowledge work.

Transformative AI Threatens Export-Driven Development

Transformative AI (TAI) systems, defined as systems that precipitate a transition comparable to the agricultural or industrial revolution, could fundamentally break this equation by reducing opportunities for global labor arbitrage across the board. If AI systems can function as “drop-in remote workers” — by the end of the 2020s, according to Amodei and other experts — they would likely do so at a much cheaper cost than humans working from developing countries. This could strip away several early rungs of the export-driven development ladder.

In this future, digital services could face rapid automation, with value moving from contractors in developing countries to AI systems operated by corporations in wealthy nations. Simultaneously, AI-driven automation could lead to increasingly capital-intensive manufacturing processes, pricing out countries that cannot afford expensive automation. Together, these dynamics could close the primary pathways available to late developers.

This could happen in three specific ways:

First, AI could prevent “leapfrogging” via digital services.

The obvious irony of this latest wave of AI automation is that digital services are being automated faster and more completely than manual labor — precisely when many developing countries were betting on them as their primary development pathway.

Africa offers several excellent examples. Kenya is launching a national business process outsourcing (BPO) policy seeking to bring in a million jobs over five years, and has been pouring resources into call centers, content moderation, and data labeling. In 2013, Rwanda launched a plan to become the “Singapore of Africa,” prioritizing the build-out of its internet infrastructure to attract foreign firms for remote digital employment.

Unfortunately, the strategy of leapfrogging via digital arbitrage appeared most promising before the release of ChatGPT. Today, those countries are beginning to discover that the digital services pathway could be closing even faster than manufacturing. Many forms of digital services could be automated within a few years of developing powerful AI systems because of near-zero marginal costs — deployment costs essentially nothing beyond API calls.

Call centers are a backbone of digital services arbitrage, yet chatbots could automate much of the industry over the next decade. Business process outsourcing — data entry, invoice handling, claims processing — is expected to see a loss of perhaps half a million jobs in India over the next three years. Even services such as content moderation and data annotation are at risk from AI automation.

Most developing countries will be left out of the AI boom in other ways, too. Much of today’s data center investment is happening in wealthy nations that already have a strong foothold in the global economy. Developing countries have increasingly few levers to pull to enter the AI value chain.

Countries betting on digital services could have significantly less time to establish themselves and climb upward, representing a total upheaval of decades of economic planning. But regions that invested in manufacturing may also find their route up the economic ladder similarly stymied by AI.

Second, AI-driven automation could raise capital requirements beyond the reach of developing countries.

Export-led industrialization has succeeded historically because bottom-rung manufacturing for products like textiles, food processing, and simple consumer goods required minimal upfront capital. Countries could enter with minimal technology and remain competitive through lower wages. The model was self-financing: profitability at each stage funded progression to the next level of development.

In the past, the technology disadvantage faced by new entrants into manufacturing sectors was modest enough that wage differentials still gave developing countries a competitive edge. Today, advanced manufacturing techniques increasingly demand greater infrastructural requirements for optimized production. These high capital requirements mean that cheap labor is becoming less important in modern manufacturing compared to factors like proximity to major markets and institutional capacity.

Increasingly powerful AI systems could exacerbate these trends by continuing to accelerate the improvement of manufacturing processes and making complex tasks easier to automate. For example, globally competitive garment factories are beginning to require AI-powered cutting systems and inventory management. Modern electronics assembly increasingly relies on automated pick-and-place machines and optical inspection systems. Many tasks previously left to human labor are now being exposed to automation due to improvements in AI.

As these cutting-edge processes become more capable, automated, and dependent on expensive capital, we could see the entrenchment of export-driven manufacturing among incumbents. Rather than bottom-rung industries transitioning to the next developing country every few decades, prohibitively high capital requirements might prevent countries with cheaper labor from competing.

The winners could be existing manufacturers, who are already investing in modernizing their infrastructure even as their labor costs rise. The losers could be newly developing countries seeking to enter increasingly competitive industries. Bangladesh is a clear example: some experts predict that it could lose as much as 60% of its jobs in the garment sector in 15 years due to these trends.

Third, AI-driven automation could disrupt the “learning-by-exporting” dynamic.

A substantial factor in the success of the historical development ladder comes from a pattern often referred to as learning-by-exporting. Research shows that developing strong export industries can have multiplicative effects throughout an economy, in large part due to the development of local human capital.

First, workers gain the skills necessary on the job to compete effectively in a global marketplace. In manufacturing, this might happen via learning effective process optimization or quality control. In digital services, this could arise from developing English proficiency or strong international relationships. As these workers move on, they disperse their skills throughout the economy, creating new economic growth and increasing human capital.

Today, it’s too early to point to concrete evidence of AI’s impact on manufacturing. However, we have already been seeing trends over the past two decades that labor employment intensity has been on the decline for manufacturing, driven substantially by labor-saving technologies. Export industries are beginning to employ fewer workers — this leads to less knowledge diffusion, weakening the learning-by-doing that made export-led industrialization transformative.

If the diffusion of powerful AI systems into both manufacturing and digital services significantly accelerates labor-saving automation, we will almost certainly see further reductions in knowledge transfers. Over time, this would have compounding impacts on the long-term development of human capital and progression along the development ladder for developing countries.

Diminishing Leverage for Developing Countries

AI automation could precipitate a crisis of labor across both developed and developing countries. Importantly, workers in AI-leading nations would not be immune from these dynamics. Early research suggests that domestic labor markets in wealthy nations may already be softening at the entry level, revealing a stark parallel between global development ladders and the domestic career ladder. Just as developing nations have historically relied on low-complexity manufacturing to climb the economic value chain, young professionals in the US rely on low-complexity entry-level tasks — “grunt work” — to build essential skills and climb the corporate structure. AI threatens to saw off the bottom rungs of both ladders simultaneously.

By automating these foundational tasks, AI could not only close the door on export-driven development in the Global South, but also hollow out the training grounds for the next generation of workers in the North. We could be on the precipice of a crisis of human capital — a world where both emerging economies and talent are denied the opportunity to develop the expertise necessary to compete.

Adaptation would be hardest for the Global South. Nations that hold a meaningful share of the AI value chain — whether through frontier model development, cloud infrastructure, or semiconductor manufacturing — retain policy options that others lack. They can, in principle, capture the productivity gains created by AI and leverage this wealth to support their affected workers. There are strong incentives for wealthy nations to maintain the stability and strength of their labor market.

Developing nations may not even have this option. Without leverage in the AI value chain, they may see an erosion of their export-led development strategies, without sufficient alternatives to support their labor forces. They may lose their competitive advantages in labor arbitrage due to AI without gaining sufficient leverage in return.

These countries are well aware of the uphill journey facing them. For example, 15 different African nations have published concrete AI strategies as of 2025, and an alliance of countries on the continent recently established a $60 billion fund to build domestic AI capabilities. Whether these efforts can generate enough momentum to vault these countries past the shaky rungs of the economic ladder remains uncertain.

—

AI may not slam the door shut for the Global South — but it will almost certainly force a strategic shift. The erosion of routine-task export advantages means that developing countries cannot rely on the industrial paths taken by earlier success stories. AI could erode the bottom and middle rungs of the development ladder, and it’s not clear that there are proven alternative strategies yet.

As we see the trends described above arising in the next decade, developing countries will have to look hard at their comparative advantages to understand the best path forward. They will have to develop their foundational digital infrastructure and find competitive niches in the emerging global AI value chain. Their most promising opportunities may begin to be more localized — domains in which individual countries have advantages in local knowledge, maintain a physical presence, or can provide context-specific data.

They may have to prioritize sectors where AI will remain a complement rather than a substitute for labor. Diversifying their economies toward more AI-resilient industries — such as healthcare, education, or tourism — may be an important component of their revised strategies.

Countries that can find nuanced opportunities to build new capabilities can and will find new routes up the economic ladder — but they need to begin confronting these risks now.

Thanks to Nick Stockton, Yolanda Laanquist, Danny Buerkli, Anna Yelizarova, Jackie Si Tou, Andrey Fradkin, and Ankit Mishra for their excellent feedback on this article.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.

Deric Cheng is the Director of Research for Windfall Trust, a non-profit focused on ensuring that the economic benefits of advanced AI are shared by everyone. He is also the lead for AGI Social Contract, a consortium of experts proposing strategies to design a new social contract for a post-AGI society.