[September 11, 2026] - In December 2023, I wrote that we should be concerned about the rapid pace of AI development and deployment in an environment with few guardrails and virtually no global enforcement mechanisms for those guardrails, even if they existed. I felt then, as I do now, that the best chance of avoiding existential threats is to quickly focus on and resource international, voluntary, consensus-based standards that establish red lines that AI systems may not cross, i.e., prohibited capabilities. I posited that we need to develop tests for compliance with those standards and enforce compliance through independent testing and regulation.
Many noteworthy events have occurred since I published that blog, and they have only reinforced my view that we urgently need standards addressing prohibited capabilities, with corresponding tests. We need an independent body to certify AI systems through objective testing, irrespective of how those systems are licensed or accessed. Instead, we have geopolitical angst over which country’s AI systems will dominate, debates over whether “open model” AI systems are inherently safer and more secure than “closed model” AI systems, and, finally, growing recognition that AI has advanced so rapidly that even corporate experts are calling for regulation. None of these trends or events change my view that we need prohibited-capability standards and independent compliance testing.
While there is considerable standards work underway in the AI space, most of it at the international level focuses on AI system management. These standards are useful, but they address different problems. Management standards address whether the organization has an appropriate process for identifying and managing risk; whereas capability standards address whether a particular system has been independently tested to determine whether it possesses a specified dangerous capability. Few of the existing standards are designed for testing, and some that are, do not have corresponding tests. Those that offer an independent testing option are voluntary and essentially result in an audit, rather than an affirmative determination, one based on a high probability, that a particular AI system is unlikely to perform prohibited actions. Such prohibited actions should be limited to a relatively small number of catastrophic capabilities.
The question is whether there is a set of prohibited capabilities that the world could agree upon, and whether technical experts from around the globe could develop technical specifications for AI systems, along with corresponding test suites and harnesses that would establish that an AI system is extremely unlikely to possess those prohibited capabilities. If we focus on a limited number of catastrophic capabilities, we should be able to get consensus on prohibited capabilities. Developing objective tests for prohibited capabilities will not be easy. But the difficulty of defining and measuring a risk should not lead us to rely exclusively on process requirements or broad principles. It should motivate the technical community to develop better testing methodologies. In addition, certification needs to be tied to a particular system/version and should expire or require reassessment after a period of timeframes or when certain changes are made.
The USG is mired in indecision over whether to regulate AI or allow innovation to proceed in an unregulated environment. Rather than providing global leadership on AI policy, proliferation, and use, the USG vacillates between concerns about foreign AI systems gaining a foothold and fears about what our homegrown AI systems are capable of. Generally, governments should not regulate emerging technologies because regulations are unlikely to address the issues that need to be addressed and may unnecessarily hamstring development and commercialization opportunities that should not have been thwarted. In my view, the EU AI Act is an example of early regulation that is both unwise and unnecessary. Some would argue that early regulation is the only option because international, voluntary, consensus-based standards will take too long. That may be true for standards addressing complex AI life cycles and management processes, but it may not be true for prohibited-capability standards.
At the same time, there is a debate over whether “open weight” AI should be encouraged or discouraged in the name of safety and security. This debate is a distraction. Model weights are only a small part of the overall AI system. Even if one has access to the model weights, one cannot reproduce the entire system or, more importantly, understand what it can or cannot do. If the debate were more accurately focused on whether “open systems” should be encouraged or discouraged, there are numerous system components to consider, and there is significant debate over what makes a system “open.” In reality, openness falls on a spectrum, from very open licensing frameworks to strictly proprietary licensing frameworks, on a component-by-component basis.
Even if all AI system’s software, model weights, data, and documentation were freely accessible to everyone, for any use with or without modification, experts still would not know whether the system is safe or secure, or whether it would act in a particular way. Robust testing is needed to determine whether an AI system possesses a particular capability, irrespective of how its components are licensed. The open-versus-closed debate is fundamentally about business-model competition. Despite the best efforts of commercial entities and open-source advocates to link the open-versus-closed debate to safety and security, the USG should not allow it to drive AI safety policy. Whether a model is open or closed cannot tell us whether it possesses a particular dangerous capability. Testing can.
If the USG, however, is interested in promoting US AI innovation, it must do so by promoting stronger IP protections for AI. In recent years, there has been an erosion of patent rights for software-implemented inventions; protection for interfaces critical to protecting IP in a system or platform has been nearly eliminated; and, more recently, the USG has supported the unlicensed use of copyrighted works to train AI models. These are merely examples of areas where the USG needs to reverse course if it wants to encourage long-term innovation. The USG needs to strengthen IP protections to achieve the right balance between promoting healthy competition and encouraging innovation.
The USG has a real opportunity to maintain and grow US technology leadership in the global AI race while also providing leadership in ensuring that AI systems cannot pose existential threats. Reversing the trend toward weaker IP protections will help ensure that innovators have incentives to develop and compete. Promoting and resourcing internationally approved prohibited-capability standards, together with independent compliance testing, would provide meaningful guardrails to reduce risks associated with all AI systems, irrespective of whether those systems are classified as open or closed.
