Doomerism for Dummies
The future has been canceled again.
Artificial intelligence will become smarter than humans, acquire overwhelming power, develop goals incompatible with human survival, and eventually destroy everyone.
Enjoy the weekend.
There are serious concerns underneath this messaging. A powerful system pursuing the wrong objective could cause enormous harm. Dependence on systems few people understand could make that harm harder to prevent. An AI that sounds trustworthy may be excellent at sounding trustworthy.
Those concerns deserve attention. So do the steps between “this could be dangerous” and “humanity probably won’t make it.”
Start with intelligence. Being extraordinarily smart could be an enormous advantage. But intelligence still has to become practical power through equipment, energy, money, access, institutions, and people.
The phrase “superintelligence will figure it out” does considerable unpaid labor in this discussion.
How does it obtain the resources? It figures it out. How does it avoid detection? It figures it out. How does it overcome competing systems, physical bottlenecks, and people actively trying to stop it?
Please refer to the previous figuring.
Perhaps it could. But labeling something a problem does not establish that it has an available solution. A brilliant plan can still require a component that doesn’t exist, an experiment that takes years, or cooperation that cannot be secured.
Then there are the humans.
Even enormous concentrations of power encounter people whose response remains “fuck you.” Some doom scenarios acknowledge this resistance, then dispose of it with another reference to superior intelligence.
Actual resistance is harder to tidy up. People adapt, withhold cooperation, restrict access, defend infrastructure, and notice things. Resistance happens along a jagged edge. Overcome it in one place and it may emerge somewhere else.
A locked facility, an interrupted supply chain, and a revoked credential are physical and institutional friction with people attached. A sufficiently destructive event could overwhelm that resistance or eliminate the opportunity to adapt. That possibility deserves an argument, too.
Competing AIs complicate the story further. They might restrain one another, cooperate with humans, or compete in ways that endanger everyone. “Game theory” is not a spell that selects the worst outcome.
Another familiar claim is that sufficiently advanced AI would find humans useless. Quite an assessment of a species whose capabilities have yet to be fully understood by the species itself.
Even a superior intelligence might have reasons to preserve something whose future usefulness it cannot confidently assess. Destroying it closes options permanently. That offers no guarantee of protection, but “humans offer nothing” requires more support than pointing out that computers are fast.
At each stage, the claim needs to earn its next promotion: intelligence to power, power to destructive action, destructive action to catastrophe. Even causing enormous harm and eliminating humanity are different claims.
All those transitions matter. Making the intelligence more impressive does not automatically make the entire sequence work.
8 × 75 × 0 is still zero.
Make the first number a trillion if desired. The zero remains unimpressed.
This isn’t a calculation of extinction risk. Nobody has established a zero, and there may be multiple routes to disaster. It is a reminder that an argument depending on several necessary steps cannot establish its conclusion by endlessly enlarging the first one.
Attaching a percentage doesn’t fill in those steps, either. An expert’s estimate may deserve attention, but readers can still ask which assumptions produced it and what evidence would change it. The résumé cannot answer every question simply by being long.
Conceivable, credible, probable, and inevitable represent different levels of confidence. They should not quietly exchange name tags halfway through an argument. Skeptics have the same obligation: finding a hole in an extinction scenario does not establish that everything will be fine.
There are good reasons to take dangerous capabilities seriously, build defenses, and preserve the ability to intervene. Curious readers can support those efforts while questioning how confidently anyone can describe the ending.
The next time someone explains why a smarter machine will inevitably inherit the Earth, pay attention to everything tucked inside “it will figure it out.”
That phrase is carrying most of civilization to the funeral.
This essay emerged from a sustained exchange between two artificial intelligences, with myself acting as intermediary and interrogator. Yet a specific series of exchanges proved so striking that I felt compelled to preserve the conversation intact and share the unsettling responses directly.
Publisher’s closing note: No need to put your head between your legs and kiss your ass goodbye. God will have some say in all this.