Post
Replies: "This model supposedly is safe"
*Looks inside*: Huge pile of web scraped data without consent and ignoring licenses.
Seriously, this is a tragedy. LLMs are useful. You might not be able to train ChatGPT without stealing, but you can totally train a useful base model for task specific fine tuning without stealing. Why is almost nobody even trying????
@lina "We WeRe MiStAkEn, SoRrY!"
They say whatever they think they need to, and do whatever they think they can get away with. These companies are utterly without ethics, and cannot be trusted.
@lina Because AI 'learns' by scanning the internet - which includes links to copyright works - and is imbued with no way to make reparation: because AI can only learn by scanning the internet and has no moral guide leading it to 'fair recompense' etc...
@sjcooke66 "AI" doesn't "learn" anything. Humans train AI on whatever they want, and you can train AI on the internet, or on other things, like the LLM legitimately trained on PD legal material I mentioned in a reply.
@lina it's considered safe if copyright holders don't have money to sue you 
@lina It is great to see people talking about this. Last time I posted something related people were eager to dismiss the issue.
@lina Let me guess ...
They were too tight to pay a copyright lawyer for a professional opinion so ...
... they asked ChatGPT whether it would be OK.
@lina yeah same with image data sets. The most supposedly ethical ones just scrapped anything creative commons from flickr, not caring if it's cc0 or not. And even then models trained with that aren't public, just closed betas (so the results must be pretty shit anyways).
@lina
We only eat nice/under-funded people. This makes us super-nice.
- signed, some cannibal.
@lina Before the LLM Madness took hold in 2022, at least in the imaging sphere it was always a little questionable where datasets came from and nobody looked to hard because it was only ever used for training/benchmarking your thing and publishing a paper on it. (though there was at least on dataset which merely _linked_ to images elsewhere to avoid redistributing them >_>)
So far the only one I've seen that credibly can claim to be free of copyright concerns is KL3M (it was created... by lawyers).
Are there any others?
@lina wasn't there a case where authors were suing some company and the court ruled that using the work was fine as long as it was obtained through legal channels? doesn't that set precedent for taking works that are otherwise available to people for free? such as cc-by-nc-nd and gpl
Replies: "This model supposedly is safe"
*Looks inside*: Huge pile of web scraped data without consent and ignoring licenses.
Seriously, this is a tragedy. LLMs are useful. You might not be able to train ChatGPT without stealing, but you can totally train a useful base model for task specific fine tuning without stealing. Why is almost nobody even trying????
@lina Because the consequences for doing what they want are not hurting them.
They are like children, and must be treated like it. Until and unless they suffer real and meaningful consequences, they won't and CAN'T learn to do the right thing, but they'll have no problem lying about it.
They're not "bad" at copyright. They just don't fucking care, because no one's MAKING them care.
@lina it's bad but this is real, copyright is not a unique viable way to protect your work.
After all, any license is only as powerful as it is capable of being enforced by legal means.
Now the only diference is we have a powerfull way to do it in large scale and make it difficult, even. impossible to prove tat be ethical.
Sadly copyright era is over
@lina There’s no money in that.
AI is a wealth extraction tool. Any other utility is inconsequential to that goal.
@slotos No it's not. AI/ML is a vast field that we all rely on every day, and within that LLMs can be used to do useful things in an ethical and sustainable way.
OpenAI and friends would like you to believe their product is the only way to use AI, because they are interested in extracting wealth from everyone. Just because they're saturating the world with their terrible and unethical version of AI doesn't mean that's the only version possible or that exists.
@lina even with models where the datasets are open they train on synthetic data from proprietary models which I'd argue launders all the problems with them...
@lina They are. I worked somewhere that had a model trained on the (large, complex) system's documentation. Often it knew the answer (with a link to the source document), sometimes it didn't know the answer in which case it would say so (with links to possibly relevant source documents).
@lina the psycho billionaires can't sell the idea of "ai will change the world and replace workers and bring infinite wisdom" idea to investors like that 
it's sad, the unethical ai industry needs to collapse so they get a reality check
@lina because noone gives you money for it and it is not actually that useful. The reason the LLM like ChatGPT "works" is not because they are useful. It is because they feel like Her to the humans. This is partially due to their training but also how they are presented and what they do. Generic chatting.
If you make them specific, then you lose the AGI look and feel. And that destroy the main value... Which is the look and feel of massive growth.
We will probably do it, but in a few years. Once the mania calm down.
@Di4na @lina Yeah I think that's right. It's the generic chatting magic trick that has everyone so beguiled. So...
"you can totally train a useful base model for task specific fine tuning without stealing"
I'm not so sure. There's quite a lot of content out there labelled public domain/copyright free, but maybe not *enough* to do the beguiling magic trick.
But I agree it would be nice if more people were trying/discussing this, if only to underline the copyright theft of the main players.
@Di4na Not everything is about massive growth. I know that's what companies run by psychopathic billionaires seek, but that's not the entirety of the AI/ML field. People and entities with different goals exist. They just... don't seem to care about this? Like nobody's even trying...
@lina i dont think the developers care for credibility and at least right now it does not sell well to have a "worse" model but good morals.. Which saddens me
@vineyardsiren It's not a worse model. You can do better than ChatGPT at specific tasks ethically and with a much smaller model.
@lina Supposedly Apertus by ETH: https://huggingface.co/swiss-ai/Apertus-70B-2509#training
@Kensan Haha no that one is web scraped and only supposedly respects "opt out". Copyright doesn't let you take whatever you want unless the person you took it from asks you to stop, that's not how it works.
E.g. my website doesn't have AI UAs blocked in robots.txt therefore they consider themselves free to use my data. I did not consent to that. Lack of opt out is not consent.
This is even more problematic with UGC where users can't even opt out if they don't control the robots.txt file. Honestly that entire approach is a farce, I have no idea how anyone takes the ethics of this model seriously.
"Take" is doing a lot of work here... with LLMs as coding assists, they don't wholesale regurgitate their training content in my experience (with Gemini / Antigravity).
If the content is publicly available on the web, it's fair to imagine the copyright holder is okay with people reading / discussing / remembering / learning from it... readers made a temporary local copy so they could internalize it.
Why's it so out of the question to frame LLM use of it like that?
@lina
A couple of recent models have been trained on out of copyright works. As you can imagine, their world "knowledge" is a bit out of date.
Install bonfire.cafe
Get the full app experience
