It’s a standing on the shoulders of giants thing. That’s how humanity have been making progress historically.

People running local LLMs with open weights abliterated models, especially with solar PV power (perhaps not as rare as you think, I’m upgrading my solar PV system in the next couple weeks for that explicit purpose), is about freedom and privacy. At some point there should be enough of them to link up into a grid over the Internet, collaborate or donate resources to distributed community projects. This is a way to allow for the “pillaged” content to pay back for itself.

Do you have qualms with public pirate libraries, like SciHub and LibGen, too? These have also been used for training LLMs.

Whoah. Have you ever thought about making some videos demoeing and explaining your setup?

A fanless thin client with Opnsense and a four-port Intel NIC.

If I fire up everything in the rack it’s several kilowatts. Probably more than a 230 V wall socket can handle.

Usually it’s way less, though right now it’s just a switch and the router/firewall, mere 57 W.

I make most of my own net power with solar PV, soon 4 kWp in total.