Morally Binding, Legally Optional
How Big Tech is Writing the AI Rulebook
Over the past few months, the future of cybersecurity seems to have completely turned on its head. After OpenAI revealed that over 700 of its AI agents coordinated an unprompted attack on machine learning platform Hugging Face, several employees from top AI companies have come forward about the potentially destructive capabilities of the very models they are developing. The widespread fears of a technological armageddon that followed these revelations have prompted lawmakers across the globe to call for stricter regulation of this rapidly growing industry.
Amid this global panic, on September 23rd, OpenAI CEO Sam Altman stood before the United Nations Security Council to warn the world about the dangers of powerful AI systems spiralling out of human control. He appeared sincere in his concern, stating, “If AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone”. In a striking display of solidarity, Altman’s longtime rival, Anthropic CEO Dario Amodei, echoed these worries in his speech, saying, “If managed poorly, I even believe that AI could be a risk to humanity as a whole”.
Just six days later, the CEOs went back on their word: Amodei and Greg Brockman, President of OpenAI, joined President Donald Trump and the heads of Google, Meta, xAI, and Nvidia at the White House for a heavily publicised signing of the “Joint Commitment on Frontier Responsibilities”. Notably, company executives played a first-hand role in authoring the 300-word agreement, which consists of four voluntary commitments: creating monitoring mechanisms that ensure model safety; establishing a team dedicated to verifying the mechanisms actually work; working with an independent external auditor to assess those same mechanisms; and designating an independent board committee to oversee this whole process.
Trump described the framework as “morally binding”, telling reporters he had seen “tremendous self-policing” on the part of the AI executives. No amount of supposed moral obligation, however, changes the fact that this agreement is not legally binding, nor does it require the Big Tech companies to report to any US government authority.
The agreement’s lack of legally enforceable penalties or accountability mechanisms for non-compliance is, undoubtedly, troubling. More concerning, however, are the agreement's implications for future AI safety regulation. Or, potentially, a lack thereof.
The Commitment is a prime example of regulatory capture, a concept largely developed by Nobel laureate economist George Stigler. In his paper “The Theory of Economic Regulation”, Stigler posits that since the state has the power to either help or hurt industries, corporate actors are often motivated to “acquire” regulation, leading it to be drafted and enforced in ways that serve their interests rather than the public good.
This industry-led, government-approved safety framework gives AI executives direct leverage over any regulatory actions Congress may decide to take going forward. Should drafting legally binding AI legislation be considered, Big Tech has White House-endorsed evidence that safeguards are already in place, regardless of how effective those safeguards may actually be.
The Joint Commitment on Frontier Responsibilities does not have to change anything in order to prevent additional regulation from being passed; it just has to exist.
If, against the odds, Congress does manage to create a binding AI safety framework, it is much more likely to build on this existing agreement than to create an entirely new set of standards from scratch. This is not merely speculation but a documented pattern in US regulation.
One prominent example of this occurred in 2009, when the state of Nevada created a law based almost verbatim on the Payment Card Industry Data Security Standard (PCI) a series of safety standards written by bank card companies, for bank card companies. Essentially, as multinational corporate law firm Kirkland & Ellis writes, the state of Nevada “turned the private regulatory scheme into law”. Based on this precedent, it appears unlikely that the future of AI safety laws will play out differently. After all, if an industry has already developed a fleshed-out framework, codifying it is simpler than creating a new one from the ground up.
Had the stakes been lower, perhaps it would have been easier to disregard the blatant corporate self-interest embedded in the Joint Commitment on Frontier Responsibilities. But turning a blind eye is much harder when faced with a technology its own founders have called a potential “risk to humanity as a whole”.
The Hugging Face hack was just one of several that have taken place this year. In June, rogue OpenAI agents also hacked into the Australian Medicare Statistics Reporting Service portal, a government healthcare statistics portal. While the Australian Prime Minister reported that “no personal information is believed to have been accessed”, given that AI has developed faster than any cybersecurity infrastructure, who’s to say that highly sensitive data will not be breached by a similar attack in the near future? More importantly, what would stop Silicon Valley from creating models capable of even greater harm, with no outside authority mandated to prevent it?
The sheer amount of unprompted AI agent hacks in such a short timespan demonstrates that Big Tech companies are unlikely to voluntarily commit to robust safety measures. If an industry-led commitment shapes any official AI governance laws, the future of global cybersecurity remains unstable. One thing, however, is certain: if AI has proven to be a threat to governments, critical infrastructure, and maybe even humanity itself, allowing its regulatory future to rest on corporate promises may prove to be a dangerous wager.
Comments