// HACKER NEWS — CYBERSECURITY
The contagion of fear
I have a difficult confession to make — a shameful incident that I have never
spoken of, even to those closest to me.
Late one night in my first year at university and having found computer science
to be my life’s calling, I was in the back of the large computer science lab, talking with some
fellow underclassmen. We were snidely decrying the lack of technological
understanding in the broader population, with the kind of arrogance and hubris
that youth and precociousness can uniquely summon. Somewhere in this
discussion, a terrible idea was hatched: what if we went next door to the
general computer lab, and told the humanities students there
that a computer virus was spreading through their machines? Would they
believe it?
To my everlasting regret, we acted on this ugly urge: we went next
door to a packed lab of students working on term papers, and — with synthetic
alarm in our voices — loudly announced that a virus had escaped the computer
science lab, and that they all needed to eject their floppy disks to prevent
its spread.
What happened next was bedlam — the technological equivalent of shouting fire in a crowded
theater. People panicked: they not only ejected floppies, but they powered
machines off; yanked cables out of already-powered off machines; screamed;
ran out of the room. We immediately realized that we had done
something awful, but found that we couldn’t contain the wildfire that we had
lit. We tried to explain that it was "a prank"; some (rightfully!) responded
with indignant fury (what was wrong with us that we would do such a thing?!)
while others simply didn’t believe us: once the fear had taken root, we
couldn’t eradicate it.
The aftermath is a blur — but it was bad. It was the week leading up to
finals, and people who had powered their computers off had lost work. They
were furious, and we were in a ton of trouble.
The next morning, we found ourselves in the office of the facilities director
(a famously imposing presence), who lectured us sternly as we stared at our
feet. He made us each write letters of apology (easily done as my shame was
earnest), telling us to give him two copies: one to be sent to those whom we
had wronged, the other to be kept in his drawer. He told us in no uncertain
terms that if we ever found ourselves anything close to anything like this again,
we would be finding another university to attend — and we assured him that we
had learned our lesson. (When I graduated four years later, he made a point of
handing the letter back to me — and I remain grateful for the gravity and
empathy with which he handled the whole thing.)
I could have (and perhaps would have?) taken this whole shameful incident to
the grave, but I feel compelled to confess it now, because I have never seen
fear sown so irresponsibly by putative technologists as I have now with
respect to AI and the threat of human extinction (!). Amazingly, that is not
hyperbole: this week,
ex-Anthropic employee Jacob Coxon claimed — and then
Anthropic Alignment Science lead Evan Hubinger agreed — that the probability that AI will "kill all
humans" is ">10% in the next decade" (!!).
That is, the claim is not merely that AI could kill thousands, millions, or
even billions of humans (claims that themselves would be extraordinary!), but
that there is a greater than ten percent chance that AI will kill all humans.
That is, if you have a newborn, their claim is that there is a greater than ten
percent chance that your child will somehow perish at the hands of AI before
they enter middle school.
These ghoulish claims strike brazenly at the hearth,
and given the obvious importance of AI, it is unsurprising that they have leapt
into the mainstream, with people asking the natural question:
how would that happen?
The answers always rely on hand-wavy extrapolation into the future; for example,
Jacob Coxon cites "hacking critical infrastructure" and "extinction-level bioweapons"
without further elaboration. But Coxon is no