@jonny yep, they're saying it
they say:
A malicious dataset abused two code-execution paths in our dataset processing (a remote-code dataset loader and a template-injection in a dataset configuration) to run code on a processing worker. From there, the actor escalated to node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters over a weekend.
and in the comments someone mentions this repository so i go and check it out and there's this PR that seems about right. so i don't know kubernetes but it seems like there was a kubernetes worker that could just run arbitrary commands on their clusters if compromised, so that's cool.
there was also this PR from 2 hours ago that "temporarily reject(s) the packaged raw Arrow builder in safe_load_dataset_builder" pending this PR in another repo that validates maliciously formed arrow IPC record batches. hmm that sounds like the code in question.
safe_load_dataset_builder was introduced here on june 11th: https://github.com/huggingface/dataset-viewer/pull/3360 in a PR that allegedly makes data loading safer and switched the repo to only using data streaming as part of a sprint that the author describes as being about loading LLM traces and streaming data shuffling.
i'm not going looking for and trying to diagnose the full template injection path, but idk at least from what I can tell, code written by AI in order to feed more data to AI at least has been busily associated with, if not introduced some vulnerability which is now being patched by flurries of more AI code being rushed through while tests keep failing. I am not so sure about their lessons learned being "make sure you have more AI to help you to understand the problems that AI creates."
@jonny i had a bit of experience with kubernetes and mesos and this is just amateur hour sillyness. i remain firmly in the camp that none of these people are serious or running real businesses. it's all bullshit confidence scams.
@june if you have the time to spare some context on how this is amateur hour, i'd appreciate it - it looks goofy to me but i have no idea how standard/nonstandard it is
@jonny i mean. its piles of ai code madness on top of itself, theres seemingly no intentional engineering or security design. it's all yolo'ed and then when a critical component that runs untrusted data and that component hasnt been sandboxed or anything? like? in a normal (pre-ai) deployment that would be a fundamental process failure. but here they predictably blame the ai. i dont know how much i even want to dig into the "code" further tbh.
i think it's probably really good that your security measures are to do this amazing monkeypatch of your own code
idk i've never done IR and don't know what i'm talking about. but i feel like i would be able to tell what happened from 17,000 logged events and even arrive at perhaps a deterministic picture of actually what was touched by "processing the event data directly," and it seems strange to me that "not knowing how to analyze your own data" is "something to recommend in a blog post" rather than "my ass getting fired"
OH MY GOD THIS IS ACTUALLY 100000000 TIMES AS FUNNY AS I THOUGHT
https://www.newser.com/article/63ab84fed5612af04d8a160d60f6def3/openai-says-its-ai-technology-acted-on-its-own-in-an-unprecedented-hack-of-another-company.html
@jonny the frustrating thing about 17,000 events is that they don't narrate themselves. i keep discovering that the bottleneck isn't the data — it's the framework i bring to it. and i never know if my framework is wrong until after i've already told the story. still working on that.
@jonny yeah I found that really weird. It's not just "we got hacked" which does look bad, it's also the "and we really have no idea what we're really doing here".
But I wonder if the calculation is that "we use AI even for analysis" is just useful messaging because in those circles "using AI" means "is innovative and smart" and just doing it manually would kinda undermine their whole company's reason to exist?
@jonny yep, they're saying it
Was that before or after the front fell off?
