dream_weasel

Chaotic good.

Our first president is finally bankrupt.

https://www.goodreads.com/book/show/54503521-a-thousand-brains

That book, not the article. I can’t recall if it makes a convincing argument about consciousness, but it has a lot to say about general intelligence. I thought the neuroscience parts were a little cumbersome, but still worth the read.

Easier said than done. That’s kind of the purpose of an individual neuron approach I think. I don’t see a way that a bulk retrain is reasonable because it takes a long time, uses a lot of resources, and also requires you to have your whole training corpus handy all the time. Your brain is not doing a wholesale rewrite for every new memory, and new memories often chain together. I don’t think you can just say “ah well, let’s update the RAG or do another supervised fine tune with LoRA or something”. At present, we just don’t really have the machinery in the models to do simultaneous learning, inference, and forward planning.

I’m not a neurologist or a philosopher so I think this may not be something I can speak intelligently about, but it seems to me that the question doesn’t really live in the function approximation, but in the gaps. Doing a single train of thought is the easy part. I think what you need is an “always online” feature to keep thinking and some way to be stateful (make memories) without constantly retraining the whole network or whole sets of layers.

This is, I think, the benefit of the parallelism and asynchrony of the “individual neuron” discussed in the book.

It isn’t, but that’s not how LLMs are built. If you want to go that route, you can read Jeff Hawkins 1000 brains.

The short and sweet is he argues you can’t get artificial general intelligence the way we make LLMs, you need to basically create neurons and let them connect and interact in novel ways.

Linear algebra in neural networks isn’t really that, despite the name.

its a it’s an epidemic / its a it’s endemic