• isleepinahammock@lemmy.blahaj.zone
    link
    fedilink
    English
    arrow-up
    7
    ·
    3 hours ago

    It need not even be rogue agents. It could simply be that modern LLMs have really democratized hacking. You used to need to be a god-tier hacker to have any chance of doing something truly damaging, like bringing down the nation’s whole air traffic control system. Those few rare people that did have those skills also had enough experience and wisdom to not use them so recklessly. And they also had a lot to lose.

    But what happens when every 14 year old kid can hack national infrastructure? Previously, the only people that could this were 40 years old with a spouse, kids, and mortgage to take care of. People with a lot to lose. Now we have to worry about national infrastructure being brought to its knees by people so young that if they’re ever caught, they’ll be tried in juvie.

    My theory is that every script kiddie can now do a level of damage previously obtainable only to late-career grey beards. (And only select few grey beards with just the right kind of career experience.) And previously, the kinds of people that actually had those rare skills were mostly using those skills to earn excellent livings. There are not that many 40-something software developers who would think taking down the nation’s power grid as prank is a good idea. Previously, the Venn diagram of “those who can hack national infrastructure” and “those who want to hack national infrastructure” had very little overlap.

    But what happens when someone who has the right mentality to become a school shooter instead decides to vibe code a campaign of national computer-based terrorism? AI alignment theorists have always worried that a powerful AI might cause damage outside our intentions, like the paperclip maximizer turning everything and everyone into paperclips. But I worry about people using these tools with deliberate blind malice. Imagine someone firing up the most powerful open-weight tools they can find and prompting them, “kill as many people as possible. Find ways to hack whatever systems you can, with the goal of causing the maximum loss of human life possible.” Imagine what a school shooter would ask an LLM to do. That’s what I worry about. We’ve seen how dangerous these tools can be even by accident. Imagine if someone directly tells them to kill innocent people.

    We’ve taken the tools previously accessible only to experienced masters and put them in the hands of people who haven’t even had the moral reasoning center of their brains fully grow in yet.