How can I find a local coding LLM that wasn’t trained on copyleft code?
It’s an ethical concern, not a legal one, so if the data collection doesn’t entirely rule out copyleft, that might be OK
Just because code isn’t copyleft doesn’t mean it does not require attribution or consent to being scraped by LLMs!
Also, I don’t think there’s any model out there that has asked for permission for scraping, and if any claim they do, they’re almost certainly liars.
In my opinion, you should ideally avoid LLMs if you can do the task yourself. It definitely has some genuine applications, especially translation, but functionally, ethically, and legally, it is not a suitable option for most people.
Just because code isn’t copyleft doesn’t mean it does not require attribution or consent to being scraped by LLMs!
I am painfully aware. For my own work, I don’t use LLMs, but I’m not convinced I’m going to be able to do that at work. I’m trying to work out the details of what that means and if it’s irretrievably immoral. It’s not looking good.
What’s wrong with copyleft code?
Nothing in itself. I just felt it was antithetical to the intention behind copyleft to train on it then share the result.
It’s a good question, I’ll think on it more.
I don’t see why it would be. Isn’t ‘copyleft’ basically: “You may modify and redistribute this code for free, but whatever you produce with it has to be copyleft as well”?
Extending that to coding agents trained on copyleft code, I would say that any code that agent writes is also copyleft. How is that anti-ethical?
You’re probably looking for explicitly public domain code, not just avoiding copyleft. That will be rare to find. Permissive licenses such as MIT share with GPL and other copyleft licenses the aspect that they require attribution, while LLMs act as an attribution stripper.
Olmo might be the closest you can get
https://allenai.org/olmo (website straight up crashes my phone)
https://huggingface.co/allenai/OLMo-7B
Olmo is trained on the Dolma dataset, “a diverse mixture of web content, scientific papers, code, public-domain books, social media, and encyclopedic materials”, so this might be a more ethical source, but I’m not sure if it’s copyleft free.
https://huggingface.co/papers/2402.00159
You could rent a cluster of GPUs and remove copyleft material and retrain
Dolma uses starcoder https://huggingface.co/blog/starcoder, which
are Large Language Models for Code (Code LLMs) trained on permissively licensed data from GitHub
so this might do. Thanks.
Only correct answer so far. Here’s the licenses included in the dataset.
So you want an LLM that says they don’t train on “copyleft” code, even if they do, so you can virtue signal to yourself without facing any kind of real inconvenience?
How about you start by clarifying what you mean by “copyleft” as that’s not a standard term, and it means different things to different people. I’ve seen it used to refer to MIT style licenses, GPL style licenses, and public domain stuff… All of which are very different things.
Where have you seen people refer to MIT licensed or public domain works as copyleft? An insane asylum?
Copyleft is a standard term. https://en.wikipedia.org/wiki/Copyleft
Just because it’s used incorrectly doesn’t mean it isn’t standard. “Ironic” is a standard term and is ironically used incorrectly all the time.
Alright mate, take your high horse elsewhere please.
