I might argue with centralized systems being the only ones that allocate resources most efficiently. You can have nearly same efficiency with decentralized models, except they are more complicated to implement, but the diversification is often worth it.
In another timeline where RAM and GPUs were still largely consumer accessible, I wouldn’t doubt there would already be a SETI-like decentralized alternative for LLMs.
I don’t know about that, mostly because the structure of LLMs at the moment make them incredibly bandwidth-hungry, and also quite sensitive to latency.
You do have some small-scale versions of that, where people link a few smaller computers together to run a larger model, but that usually exists in the form of things like connecting machines via USB4/Thunderbolt, or Oculink, which are much faster and lower latency than a typical internet connection.
There is something a bit like that, with the Kobold AI Horde, but that is still people hosting an entire model and sharing that over the internet, rather than something like Folding/SETI@Home, where you’re doing part of the actual calculations yourself.
I was actually thinking more like participants who can offer computational power during periods of no use, accrue reputation the more they contribute, and can use that reputation to offload computation to the cloud. If your PC was not powerful enough to run local LLMs, it would need to contribute a lot more, or perhaps it would just be limited to clouds of people running powerful enough GPUs for the local LLMs being used so they could get an added benefit to leaving it running at night.
An LLM model running over across multiple PCs across the Internet would not really be workable, no, but their product is.
I might argue with centralized systems being the only ones that allocate resources most efficiently. You can have nearly same efficiency with decentralized models, except they are more complicated to implement, but the diversification is often worth it.
In another timeline where RAM and GPUs were still largely consumer accessible, I wouldn’t doubt there would already be a SETI-like decentralized alternative for LLMs.
I don’t know about that, mostly because the structure of LLMs at the moment make them incredibly bandwidth-hungry, and also quite sensitive to latency.
You do have some small-scale versions of that, where people link a few smaller computers together to run a larger model, but that usually exists in the form of things like connecting machines via USB4/Thunderbolt, or Oculink, which are much faster and lower latency than a typical internet connection.
There is something a bit like that, with the Kobold AI Horde, but that is still people hosting an entire model and sharing that over the internet, rather than something like Folding/SETI@Home, where you’re doing part of the actual calculations yourself.
I was actually thinking more like participants who can offer computational power during periods of no use, accrue reputation the more they contribute, and can use that reputation to offload computation to the cloud. If your PC was not powerful enough to run local LLMs, it would need to contribute a lot more, or perhaps it would just be limited to clouds of people running powerful enough GPUs for the local LLMs being used so they could get an added benefit to leaving it running at night.
An LLM model running over across multiple PCs across the Internet would not really be workable, no, but their product is.