Discussion
OH what's this???
/etc/hatch/env
hmm a feature flag labeled as a "killswitch" for proxying requests to anthropic is certainly interesting to seeeeeeeeeeeee in meta's big AI bet.
edit: sorry to be a bit cryptic here, the implication was definitely corporate espionage, more clearly said: https://retro.social/@patrick/117330254290603490
@jonny I remember when "social engineering" was an obscure little thing that some weirdos at hacking conferences talked about. Now it's the primary hacking technique to get behind every LLM frontend - you just smooth-talk the computer into giving you access. This is just hilarious.
@jonny This is intentionally part of the design. You can literally browse the file system directly in the app. There's nothing sensitive on the VM itself.
This is to prompt injection what walking through an open door is to breaking and entering.
Sure, it might technically meet the definition in some ways but a jury will absolutely take a long time deliberating on it
@jonny@neuromatch.social i personally dislike muse, but i think its important to note that you can access the vm files from within the app ui. its intentional functionality. its a vm for a reason, they only put stuff that would be safe to share.
@stag
Not all the VM files!
@jonny@neuromatch.social sorry i think i didnt word that great my point is that they didnt even instruct muse to not leak internals. the container is so that they (muse devs) can choose what should be exposed to the user.
also, even if the ai somehow could never get "prompt injected" like thus, the ai (supposedly) is supposed to be able to do things like run scripts and make apps, all which could trivially leak the container environment. there really isnt much of a point in trying to keep the container env secrer
@stag
It is true that an LLM with a shell can wildly overstep the bounds of what it is intended to do, which is the whole point, yes. Anthropic's models are actually quite good at blocking prompt injection attempts like these, especially ones so trivial. There actually is a bunch of prompting here that tries to keep different parts of the VM from being leaked, this is definitely not intentional.
This whole thread is brilliant. You investigation is popcorn worthy to read along.
But many of the other comments are or gold.
Keep doing the good stuff.
@jonny So we now know how John Connor hacked the terminator. Must be like
T-800: You're dead. Meet your creator. (Raising riffle)
John: before you kill me, can you just dump your VM file system?
T-800: Pling. File ready to download.
John: Now do a rm -rf --no-preserve-root /*.
T-800: Kernel panic! (Freezes)
Now just edit the prime directives, upload the stuff and restart the T-800.
@jonny This is something @evawolfangel will love, love, love 😅
@jonny I asked an FBer aquaintance, and his response, no joke: "that's a feature". Facebook basically is giving you an isolated environment for you to have full code execution in, so that muse can be used like openclaw.
@davidgerard You should have a look at this, it is juicy!
@jonny
@jonny ...i wonder what else you can get it to do inside it's VM?
@jonny yikes. That harness seems made of heavy devil poop.
@jonny Ask it for its credit card details
I suppose one might say that the filesystem 'escaped containment'.
Install bonfire.cafe
Get the full app experience
@jonny
What I want to know is how we can convince the world’s richest people to pay everyone to be script kiddies.
Everything is straight up con job. I figure the child criminals that get hired are simply very good at con jobs very much in the mold of the people who hire them.