Who gets to warn us the world is ending?

Not everyone wants to rule the world, but lately it seems like everyone wants to warn that the world could end.

On Tuesday, the Bulletin of the Atomic Scientists unveiled its annual year Resetting the Doomsday Clockwhich is intended to visually represent how close the organization's experts are that the world is about to end. The hands were set at 85 seconds to midnight, four seconds closer to 2025 and closer than ever to 12.

The day before, Anthropic CEO Dario Amodei — who might as well be the philosopher king of the field of artificial intelligence — published a 19,000-word essay titled “The youth of technology.” His conclusion: “Humanity has been given almost unimaginable power, and it is deeply unclear whether our social, political and technological systems have the maturity to wield it.”

If we fail this “serious civilizational challenge,” as Amodei put it, the world could well be heading toward pitch black midnight. (Disclosure: Future Perfect is partially funded by the BEMC Foundation, whose primary funder was also an early investor in Anthropic; they have no editorial influence over our content.)

Like me said earlierit's boom times for down times. But examining these two very different attempts to communicate existential risk—one a product of the mid-20th century, the other a product of our own uncertain times—raises a question. Who should we listen to? The prophets screaming at the gates? Or the high priest who also runs the temple?

The Doomsday Clock has been with us for so long – it was created in 1947, just two years after the first nuclear weapon burned Hiroshima – that it's easy to forget how radical it was. Not just the watch itself, which is perhaps one of the most iconic and impactful symbols of the 20th century, but also the people who made it.

The Bulletin of the Atomic Scientists was founded immediately after the war by scientists like J. Robert Oppenheimer—the very men and women who had created the bomb they now feared. This gave their warnings unprecedented moral clarity. In a moment of uniquely high level of institutional trustHere were people who knew more about how the bomb worked than anyone else, desperately telling the public that we were on the path to nuclear annihilation.

The Bulletin scientists had the benefit of reality on their side. After Hiroshima and Nagasaki, no one could doubt the terrible power of these bombs. As my colleague Josh Keating wrote earlier this weekBy the late 1950s, dozens of nuclear tests were being conducted around the world each year. That nuclear weapons, particularly at this moment, posed a clear and unprecedented existential risk was essentially undeniable, even to the politicians and generals who built these arsenals.

But the very thing that gave the Bulletin scientists their moral credibility—their willingness to break with the government they once belonged to—cost them the only thing they needed to end these risks: power.

As striking as the Doomsday Clock remains as a symbol, it is essentially a communication device used by people who have no control over the things they measure. It is a prophetic speech without executive authority. When the bulletin warns about it, as it did on Tuesday New START Treaty expires or that The nuclear powers are modernizing There's nothing you can do about it except hope policymakers – and the public – listen.

And the more diffuse these warnings become, the harder it becomes to hear them.

Since the end of the Cold War removed nuclear war from the agenda – at least temporarily – the calculations behind the doomsday clock have expanded to include climate change, biosecurity, the decline of the U.S. public health infrastructure, and new technological risks such as “Mirror life“, artificial intelligence and autocracy. All of these challenges are real, and each in its own way threatens to make life on this planet worse. But taken together, they cloud the frightening precision that the clock promised. What once seemed like clockwork turns out to be conjecture, just another warning among countless others.

Even more than most AI leaders, Amodei was this often compared to Oppenheimer.

Amodei was initially a physicist and scientist. Amodei did important work on the “scaling laws” This has also helped unlock powerful artificial intelligence Oppenheimer conducted critical research That helped pave the way for the bomb. Like Oppenheimer, whose true talent lay in the organizational skills needed to run the Manhattan Project, Amodei proved extremely capable as a business leader.

And like Oppenheimer – at least after the war – Amodei wasn't shy that he used his public position to warn unequivocally about the technology he helped develop. If Oppenheimer had had access to modern blogging tools, I guarantee you he would have produced something like The Adolescence of Technology. albeit with a little more Sanskrit.

The difference between these numbers lies in control. Oppenheimer and his fellow scientists almost immediately lost control of their creation to the government and military, and by 1954 Oppenheimer himself had lost his security clearance. From then on, he and his colleagues would largely be outside voices.

Amodei, on the other hand, speaks as the CEO of Anthropic, the AI ​​company that currently exists We are doing more than anyone else to push AI to its limits. When he creates transformative visions of AI as potentially “a land of geniuses in a data center” or plays out catastrophic scenarios ranging from AI-generated bioweapons to technologically enabled mass unemployment and wealth concentration, he speaks from the temple of power.

It's almost as if the strategists making plans for nuclear war are also fiddling with the hands of the doomsday clock. (I say “almost” because of an important difference: While nuclear weapons promised only destruction, AI promises big advantages and terrible risks alike. That's perhaps why it takes you 19,000 words to formulate your thoughts on this.)

All of this leaves open the question of whether the fact that Amodei has such power to influence the direction of AI gives his warnings more credibility than those from outside, like the Bulletin's scientists – or less.

While the bulletin model has integrity, it is becoming less and less relevant, particularly for AI. The nuclear scientists lost control of nuclear weapons the moment they worked. Amodei hasn't lost control of AI – his company's release decisions still matter enormously. This makes the Bulletin's outsider position less accurate. You cannot effectively warn about AI risks from a position of pure independence, since the people with the best technical knowledge are largely in the companies that develop them.

But Amodei's model has its own problem: the conflict of interest is structural and inescapable.

Every warning he issues is accompanied by the phrase “But we should definitely keep building.” In his essay, he explicitly argues that it is “fundamentally untenable” to stop or significantly slow AI development – ​​at least when it comes to Anthropic Don't build powerful AI, someone worse will. That may be true. It may even be the best argument for why security-conscious companies should stay in the race. But it's also, conveniently, the argument that allows him to keep doing what he's doing, with all the immense benefits that can bring.

This is the trap that Amodei himself describes: “There is so much money to be made from AI – literally trillions of dollars a year – that even the simplest measures make it difficult to overcome AI’s inherent political economy.”

The Doomsday Clock was designed for a world where scientists can leave the institutions that created existential threats and speak with independent authority. We may no longer live in this world. The question is what we will build to replace it – and how much time we have to do it.