Post by @jonny@neuromatch.social

RE: https://neuromatch.social/@jonny/117325293459803287

This is I think the most consequential thing I found in the muse VM, or at least the thing that would be most worth following up on. there are no other plain string matches I can see for what uses it, I'm looking in the binaries now, but the comment seems to indicate pretty clearly that a) meta uses anthropic's models internally, presumably to develop the thing (the code, comments, and skill text reek of claude), but also b) they use some kind of internal proxy that intercepts and proxies all requests to anthropic's API. the only reason I can think of for doing that would be model distillation.

2 Boosts

0 Likes

tkissing

@tkissing@mastodon.social 41 min. ago

@jonny At my job we have an internal proxy that all model usage has to go through as well and the main supposed rain is compliance, e.g. it supposedly routes all prompts thru something that prevents PII or trade secrets from actually going to an anthropic-hosted model. My which magic it does that is beyond me (but I fear it's using a small self-hosted model for that) - unlike many other tools we have internally, I can't get the source for this one, so I can't look...

xXx_K1R4_xXx t3h FoXXXDerG oF d00m🏳️‍⚧️

@kirakira@furry.engineer 1 hr. ago

@jonny all is fair in love and ai, it would seem

jonny (nonvenomous)

@jonny@neuromatch.social 2 hr. ago

As always, the usual caveat, I have no special expertise and am a total noob. I try to communicate degree of certainty but sometimes don't get that quite right, and so in this case I want to be clear that it looks like corporate espionage to me but the error bars around that are humongous.

Jonas Schäfer

@jssfr@zombofant.net 1 hr. ago

@jonny My razor says that they use anthropic's API as backend and had *at least* the smarts to not put their API key on the VM (hence, go through an internal reverse proxy which injects the API key).

That still means that their AI product is just a front to whatever Anthropic model they're using.

jonny (nonvenomous)

@jonny@neuromatch.social 58 min. ago

@jssfr the comment says that in production this disables the proxy, and it makes it seem like it rewrites it vm-wide with e.g. a hosts entry - if they were just proxying the production prompts, then i don't think there would be a flag. it's sort of perplexing, like did they develop the container image from within the container? not sure what it means, but i think if they were just proxying all the muse requests to anthropic that it would be very easy for anthropic to see that, they have published breakdowns of them detecting things that would be on a much smaller scale than muse. so a development-time proxy makes more sense to me, even though i haven't been able to positively confirm the use of this flag anywhere.

Jonas Schäfer

@jssfr@zombofant.net 49 min. ago

@jonny Ah! Now I get it.

Did you see the part about "grepping for the full name doesn't work because it's assembled at runtime"? Maybe you need to search for underscore-separate fragments of it, case-insensitively maybe even.

1 more reply

Viss

@Viss@mastodon.social 2 hr. ago

@jonny muse is just claude in a groucho marx mask?

jonny (nonvenomous)

@jonny@neuromatch.social 2 hr. ago

@Viss
Muse appears to have at least been written by Claude, at least the prose style and some of its quirks are all over the place, but that's increasingly hard to tell due to the degree of cross-pollination and distillation of all the models. And if so, it appears as if the writing of muse by Claude contributed to some paired prompt/response distillation dataset. Or else why would this comment be here? Why would the flag be here?

The comment is LLM comment spam directed towards itself, where presumably it removed it at one point and it broke something and was prompted to restore it, at which point LLMs, and especially Claude, will add a spurious comment explaining why something shouldn't be removed that includes too much detail that leaks from the prompting context.

Viss

@Viss@mastodon.social 2 hr. ago

@jonny a literal forensic firehose

Viss

@Viss@mastodon.social 1 hr. ago

@jonny though if theres a built in proxy to claude, i guess the next question becomes how to tell if youre really just talking to claude and metas ai doesnt exist?

1 more reply