Everything is a Remix (2023 Edition)

>LLM-creation ("training") involves detecting/compressing patterns of the input. There's a pretty compelling argument that this is essentially what we do, and that what we think of as creativity is just copying, transforming, and combining ideas. LLMs are interesting because that compression forces distilling the world down into its constituent parts and learning about the relationships between ideas. While it's absolutely possible (or even likely for certain prompts) that models can regurgitate text very similar to their inputs, that is not usually what seems to be happening. They actually appear to be little remix engines that can fit the pieces together to solve the thing you're asking for, and we do have some evidence that the models are able to accomplish things that are not represented in their training sets. Kirby Ferguson's video on this is pretty great: ▶ this video

in “AI assistance when contributing to the Linux kernel” · View on HN

Anytime these discussions pop up, it's good to link to the documentary "Everything is a Remix" (here is the 2023 updated version: ▶ this video " ) It highlights how essentially everything in culture is a remix of some previous version of the culture (or some aspect of it). E.g. even language is largely a remix of earlier languages with some occasional creation of new words. Also, while true that there is definitely an oligopoly, it's also clear that the Long Tail is both huge AND easy to access. In the past, you could only see movies by going to the movie theater and there were only a limited number of theaters. Now, you can both create and consume media in niches as small (or combined) as you want.

in “Pop culture has become an oligopoly (2022)” · View on HN

When I cover generative AI in my Ethics in AI lecture, one of few soapbox opinions I give is that GenAI is doing essentially what people do - copy others. Picasso has a quote about "Good Artists copy, Great Artists steal", which doesn't mean try to pass Lario and Muigi off as your own, but rather that great artists are able to take aspects from other works (also called 'inspiration') without being caught. My personality is a combination of elements taken from Jim Carrey, Robin Williams, and King of the Deathmatch Mick Foley. I like making vector graphics based on pictures. I have a folder on my computer called "Website Ideas" that's just screenshots of UIs that I've come across that I really like. I also point to a YouTube video by Kirby Ferguson "Everything is a Remix" [1] which talks about how so much of our collective culture stems from copy. It's a great video if you have an hour. When Little Timmy crayons a copy of Mario, we congratulate him for his creativity. Is it unique, one of a kind art? Well Timmy made it, but he didn't think up the original idea of a video game plumber. I give this view to GenAI right now - it's not capable of achieving that "next step" in "original design", but its performing like a novice artist/musician, it's mimicking what it sees. [1] ▶ this video

in “Things are about to get worse for generative AI” · View on HN

Everything is a Remix: ▶ this video Either all information is stolen, or none is. Can't have it both ways.

in “Amateur may have cracked Linear A, a 120-year-old puzzle” · View on HN

From the article: > parts of the training data can be memorized in whole or in part by current generative models Humans do the same thing... >can therefore be generated again with suitable prompts by end users and thus reproduced Personal copies (even verbatim) are not disallowed in most countries AFAIK, the concern is regarding distribution which is a separate issue. I would have thought it would be fair use: ▶ this video Since the derived work is merely inspired by the original, just like a human being uses inspiration to create new things, whether consciously or not (hence the video "everything is a remix"). The only debate IMO is just how much is being "copied" exactly, and how obvious it is.

in “AI Training is Copyright Infringement (under European law)” · View on HN

Nothing in permissive licenses prevents you from adding a useful feature and licensing the result under the GPL. If your fork is better for users, it'll catch on. I really appreciate the permissive licensing in the Rust ecosystem as it greatly eases the task of writing code for pay. While the finished product may have a commercial license, I often find bits to improve in the permissively licensed parts and contribute them back upstream. Customers seem perfectly fine with this arrangement. Tough to do the same with the GPL - even LGPL'd libraries complicate contract terms and distribution a little by comparison. With the huge productivity increase LLMs provide for writing code, it seems to me that we're rapidly entering an era in which libraries and tools for everything are available in every language, and under every license, which seems like a good thing. It is nice not to feel limited by one's language choice or work environment. I did a fair amount of work on the RepRap project, which is mostly GPL'd, and that worked out OK, but there have definitely been opportunities lost over the last 15 years or so due to license constraints which more permissive licenses would have allowed. Finding a balance which helps developers put food on the table while writing open source code also seems like a good thing. The GPL is great. I think there are important projects which really benefit from the strong incentive it provides to share. But there's definitely room for more than one way t

in “Swapping GNU coreutils for uutils coreutils on Gentoo Linux” · View on HN