Post by @stag@sharkey.world

@GhostOnTheHalfShell@masto.ai 17 hr. ago

What I want to know is how we can convince the world’s richest people to pay everyone to be script kiddies.

Everything is straight up con job. I figure the child criminals that get hired are simply very good at con jobs very much in the mold of the people who hire them.

Alpaca Male

@alpacamale@social.cologne 23 hr. ago

@jonny I remember when "social engineering" was an obscure little thing that some weirdos at hacking conferences talked about. Now it's the primary hacking technique to get behind every LLM frontend - you just smooth-talk the computer into giving you access. This is just hilarious.

Daniel Lo Nigro 🇦🇺

@dan@toot.d.sb yesterday

@jonny This is intentionally part of the design. You can literally browse the file system directly in the app. There's nothing sensitive on the VM itself.

jonny (nonvenomous)

@jonny@neuromatch.social yesterday

@dan sure yeah i'm sure they intended me to be able to torrent and mine monero on their free consumer AI chatbot product.

Daniel Lo Nigro 🇦🇺

@dan@toot.d.sb 24 hr. ago

@jonny okay that part is probably not expected lol

George B

@gbargoud@masto.nyc yesterday

@jonny

This is to prompt injection what walking through an open door is to breaking and entering.

Sure, it might technically meet the definition in some ways but a jury will absolutely take a long time deliberating on it

Mark Shane Hayden

@msh@coales.co yesterday

@jonny boy howdy when #Meta does Open Source they really go hard even when it is on accident

stag

@stag@sharkey.world yesterday

@jonny@neuromatch.social i personally dislike muse, but i think its important to note that you can access the vm files from within the app ui. its intentional functionality. its a vm for a reason, they only put stuff that would be safe to share.

jonny (nonvenomous)

@jonny@neuromatch.social yesterday

@stag
Not all the VM files!

stag

@stag@sharkey.world yesterday

@jonny@neuromatch.social sorry i think i didnt word that great my point is that they didnt even instruct muse to not leak internals. the container is so that they (muse devs) can choose what should be exposed to the user.

also, even if the ai somehow could never get "prompt injected" like thus, the ai (supposedly) is supposed to be able to do things like run scripts and make apps, all which could trivially leak the container environment. there really isnt much of a point in trying to keep the container env secrer

jonny (nonvenomous)

@jonny@neuromatch.social yesterday

@stag
It is true that an LLM with a shell can wildly overstep the bounds of what it is intended to do, which is the whole point, yes. Anthropic's models are actually quite good at blocking prompt injection attempts like these, especially ones so trivial. There actually is a bunch of prompting here that tries to keep different parts of the VM from being leaked, this is definitely not intentional.

1+ more replies

Pseudo Nym

@pseudonym@mastodon.online yesterday

@jonny

This whole thread is brilliant. You investigation is popcorn worthy to read along.

But many of the other comments are or gold.

Keep doing the good stuff.

Bernd Paysan R.I.P Natenom 🕯️

@forthy42@mastodon.net2o.de 2 days ago

@jonny So we now know how John Connor hacked the terminator. Must be like

T-800: You're dead. Meet your creator. (Raising riffle)
John: before you kill me, can you just dump your VM file system?
T-800: Pling. File ready to download.
John: Now do a rm -rf --no-preserve-root /*.
T-800: Kernel panic! (Freezes)

Now just edit the prime directives, upload the stuff and restart the T-800.

Fabian Transchel

@ftranschel@norden.social 2 days ago

@jonny This is something @evawolfangel will love, love, love 😅

echarlie

@notecharlie@social.bigcavemaps.com 2 days ago

@jonny I asked an FBer aquaintance, and his response, no joke: "that's a feature". Facebook basically is giving you an isolated environment for you to have full code execution in, so that muse can be used like openclaw.