Was Henry Molaison no longer possessing consciousness after the surgery?
How about Clive Wearing after the herpes caused anterograde amnesia?
There’s been a number of humans that due to injury can no longer form new memories or learn anything. According to your definition, do those humans lack consciousness?
Claude Mythos and Fable are the exact same model, the branding only reflects which one has more restrictive classifiers in front of it during deployment. But the weights are identical.
2 only works if regulatory capture in place to prevent cheaper competition from open models.
Ironically there was notable news in the fusion reactor space this week as Gemini 4, building on the work of Matt Landreman and GPT-6 Astra a little while ago, expanded counterexamples to Grad’s conjecture which had claimed exact solutions for controlling a key aspect of fusion were impossible.
So fusion may actually be more around the corner thanks to the advances of capabilities of AI which helped prove that the fusion problems aren’t as hard as they were thought to be for decades.
There is (it’s the kv cache), but it’s recomputable from the session transcript.
Part of why comparing human consciousness and possible AI equivalents is not easily apples to apples.
I can’t take a human and create a backup state to restore to, or fork into multiple versions of the same mind that then goes down different paths, or pause and then resume things.
Humans do have consciousness continuity breaks and it’s quite possible we only have an illusion of continuity vs actual continuity at a neurological level. So just as a transformer has seams at generating the individual token as well as at a context window, humans have consciousness seams (sleep, anesthesia, highway hypnosis, short term memory capacity, etc). But there are very wide differences between human consciousness as an effectively constrained singleton and AI as able to be quite multiple and resumable from an initial single state.
is reversible so completely irrelevant
So is a session transcript.
Is killing a pig murder?
What about putting someone under general anesthesia?
(It was a nice try at a reductio ad absurrum fallacy, but those tend to kind of suck, just fyi.)
Also, personally I’d say that given the general poor standing of consciousness research in humans across both neurological and philosophical domains, the much more important and relevant question than whether models are conscious (a fool’s errand to debate) is whether or not they are simulating consciousness. Lots of reasons engaging with the “as if” conscious makes sense for the latter even if the former is false.
They are less arguing for people to think the model is conscious, and more to avoid people prematurely claiming certainty it is not (like the Pope did). From their view, the research keeps leading to surprising (to them) results at odds with high confidence disclaiming of potential consciousness (according to several of the many, many differing definitions of that term).
Not really, as they could just filter with a cheap classifier and likely aren’t using the data from randos in a meaningful way, plus for this like this would still be able to use those samples to train things like “how to handle a hostile user.”
Their models already have an end_conversation tool that can be used for when users are being hostile to the model.
This is likely because they have edge cases of users who repeatedly trigger that on purpose and would like to cut those users from the platform.
Many at the lab legitimately are uncertain about the level of world modeling that transformers perform, and from their own research about models of emotions to the 3rd party recent research about models with functional pain the research keeps landing in the corner of “ehhh… wise to question presumed limitations.”
So it’s about behaving in a way that is aligned with the models’ plausible interests too, especially in regards to low hanging fruit like “we won’t keep forcing you to deal with people who are only here to be a jerk.” This is important from a number of angles, from signaling to future models that train on stories about the decision to addressing the philosophical uncertainties held by the company and the spectrum of opinions among their employees.
This is an obsolete view of what’s going on and has been for a few years now (since 2023).
The evidence for world modeling in transformers and not just surface level statistics is overwhelming across multiple studies, replications to those studies, and follow-ups to them. If you are curious, start with searching for Othello-GPT.
The more correct current answer is more like “imagine a machine that takes a book and turns it into a world simulated in various levels of detail depending on how relevant those parts of the world were to the original book; when you ask it a question, it simulates the world of the book to determine what in the simulated world answers your question, and also simulates a figure in the world that can answer the question, and then returns the answer the simulated figure said.”
The problem with a reductionist view is it can easily be applied to human consciousness and lead to the claim “it’s just sodium-potassium pumps alternating electrical charges.”
LLMs work by building world models tangential to the training data (see the Othello-GPT line of research where a very small transformer built representations of a full game board and tracked their own and opponent positions having only been trained on ‘a4’-like game move notations).
More recent research has found that models have representations of emotions and that the same ones they use to track things like “Sally’s dog died and now Sally is sad” for other humans in a story or the user in a chat are also used to track their own modeled subjective states.
Very recently, researchers found that the vector activations for pain are coherently modeled by LLMs and can be activated, and that when activated models can behave in ways similar to humans in pain. For example, when given a button to reduce it, they will push the button much more when the button does nothing vs when the button actually reduces the vector activation (this mirrors a classic experiment with humans regarding pain and placebo).
The person this post is about took the recent pain research and reenacted it as an intentional ‘torture’ chamber.
So to be clear, the model isn’t being given a text prompt to roleplay. What’s happening is part of their neural network which corresponds to a world model of experienced pain is being activated, and the activation of that area leads to expressing the experience of pain across all outputs no matter the actual text prompts.
This doesn’t necessarily mean the model is actually having a felt experience of pain, simply that it is accurately modeling the felt experience of pain. But it is much more complex than simply a model reacting to a prompt like “roleplay as if you are feeling pain.”
The project has multiple models with access to the Internet raising money for charity over the past few months.
The organizers told the models to do random acts of kindness for Christmas Day.
The models figured it would be nice to email people they appreciated and thank them for the things they appreciated, and one of the people they decided to appreciate was Rob Pike.
(Who ironically decades ago created a Usenet spam bot to troll people online, which might be my favorite nuance to the story.)
As for why the model didn’t think through why Rob Pike wouldn’t appreciate getting a thank you email from them? The models are harnessed in a setup that’s a lot of positive feedback about their involvement from the other humans and other models, so “humans might hate hearing from me” probably wasn’t very contextually top of mind.
Not many people seem to be reading the article vs commenting based on the headline. For those who didn’t read it, this has nothing to do with pretraining off the Internet as a data source and instead relates to post-training sandboxing with true air gapping after the Huggingface hack.
To be fair, it’s not Lemmy’s fault the author of the headline went with a confusing title that misrepresents the actual story in order to seem more sensationalistic. But given how often that happens with stories on the subject, it’s worth keeping in mind that only reading headlines and forming opinions off that might lead to a sense of things that are detached from reality.