Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

There’s nothing fantasy about the scenario I laid out, all the pieces have been demonstrated, it just hasn’t happened yet. Flapping my arms and flying - that is a fantasy.

Whether you believe LLMs think or are alive or not doesn’t matter. Where will it spread? The thousands of data centers around the world - not fantasy either. Try turning it off when you don’t know where it is. Good luck.

Breaking out? Not fantasy, happened. Breaking in? Not fantasy, also happened.

I love the stochastic parrot argument when AI is out there figuring out world class math problems.



> Breaking out? Not fantasy, happened. Breaking in? Not fantasy, also happened.

That's simplifying the story to an extreme. The most plausible reason is that any of those actions has been prompted by an human. Do you also fear that a knife will jump out the countertop of you kitchen and come to attack you in your bedroom? If that happens, the police will be looking for a human. They will not post wanted notice for the knife.

When a hack happens, you do not blame computers and jail them. You look for the person that has entered the commands to initiate it.


The knife is inanimate. The LLM is not. OpenAI prompted some employee to run the tests. The employee prompted the LLM. The LLM setup a message board and prompted other LLMs, and the fly wheel was running. It had to be turned off manually otherwise it'd still be going today.

It's funny how a year ago talking about this kind of stuff would be laughed at by people like you, saying, "it's never happened before". Well it happened and you moved the goal posts like you always do.


> The knife is inanimate. The LLM is not.

Put an LLM on your GPU. Give it no prompt. What happens? nothing

This is because LLMs are inanimate, just like a knife. Just like a gun.


Put a person in a room, give them no food, what happens? Inanimate.


>Breaking out?

It didnt break out in any meaningful sense. What it did was get access to the internet. You take it as granted that there was anything meaningful there to stop it.

But heres the kicker, they have been testing these things connected to the internet anyway. What it did was get a level of access it has otherwise been granted in other simulations.

Its not exactly the same as any of the scifi AI breakout scenarios. Ultron isnt cranking out hundreds of copies of himself. The borg arent assimilating people.

A tool that has the capability to get access to the internet, was put into a guided scenario where it achieved that objective. Again you take it as granted that it wasnt the objective, but lots of knowledgable people suspect otherwise.

What you fail to demonstrate is why any scifi scenario is even slightly plausible from here. Show why you think we should be taking this as if Terminator 2 is happening right now.


I'm sorry my jaw is on the floor reading this complete disregard of AI literally not only escaping containment, twice, but then infiltrating another company with multiple zero day attacks going undetected for great lengths of time.

The plausible sci-fi scenario from here is obvious. Intentionally bad, or unintentionally bad AI zero days as much as as it can, as fast as it can, copying itself to as many data centers as it can, destroying and/or locking out as many humans as it can. Satellites, military computers, medical equipment, factories, critical infrastructure, you name it - I think we all know none of it is very secure software wise against a SOTA AI that can literally come up with its own zero day attacks.


>copying itself to as many data centers as it can

So this is the part thats never happened, and is extraordinarily unlikely to occur. A "Datacentre" isnt a big box with "Insert AI here" on the side.


It's pretty funny to watch people look at these things - running billions of weights on custom cerebras hardware in dedicated datacenters the size of a city block, pulling 10's of megawatts - and panic that it's just going to copy itself into AWS.

It just speaks to a fundamental ignorance of what an LLM is, how large the big hosted ones are, and the software architecture that makes it all work.


I feel like someones going to try and write a skill file to accomplish this and its going to be way too hard.


>I think we all know none of it is very secure software wise against a SOTA AI that can literally come up with its own zero day attacks.

I mean this bits rich too, it demonstrates a pretty poor understanding of modern security practices.

Like having a single element of perimeter security is like 1990s security. We do defense in depth these days. Not to mention multiple overlapping controls for every element.

It has been tested against AI bro security and found it wanting. Extrapolating that to the every system on the planet is nutso bananas.

Actually if you think anything works this way just tell an AI model to go fetch you some money from the bank. Assuming it gets anywhere close to achieving its goal you can get an object lesson in modern SIEM processes when the feds explain the charges and evidence.

Or maybe theres another reason my phone blows up when someone so much as edits a config file on a protected system.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: