Artificial-intelligence systems such as large language models (LLMs) can exhibit behaviour that seems strikingly intelligent. This has led many people, including some experts, to wonder whether these systems could have some form of consciousness — the capacity for subjective experience. However, we think that existing AI systems are not conscious, and that, along current trajectories of development, they are not likely to become conscious. Greater clarity about whether AI could become consciousness will be crucial as society grapples with the challenges posed by this increasingly powerful technology.
It’s a computer program and some data - nothing more.
I’ll happily treat it anyway I see fit at the time - including powering it off without loosing any sleep over it.
Nobody can even agree on a concrete definition of consciousness so I’m not going to bother reading these ‘reasons’.
Wait are you implying you think LLMs could be conscious? Fuckin hell we’re cooked.
Define ‘conscious’.
Sapient, salient, sentient, self-aware. An LLM can’t be any of those things any more than an abacus can be. But I’m sure you’re just gonna devolve into philosophical rhetoric about nature of consciousness that still isn’t actually relevant but that will allow you to apply it to your misunderstanding of how LLMs function. This thread is apparently an AI bro brigade and I’m not interested in engaging with a bad faith group.
you seem more bad faith than the group
Says the AI bro. Seriously your comment history is public, your whole account just glazes AI.
Since you want to look at my comment history, consider actually reading next time. I am as far from an AI bro as you can be, but it’s easier for you to think so instead of considering arguments.
Read the thread. Everyone defending the idea that LLMs can be conscious are spiraling off into theory of mind that isn’t actually relevant or applicable. One dude’s whole argument is based off his totally unproven assumption that consciousness is a property of the physical structure of brains which is both limited and misguided. There are no serious arguments here, just philosophizing.
Nobody can even agree on a concrete definition of consciousness
Not only is that false, it doesn’t make logical sense…
Lots of people agree on a different definitions of consciousness.
And most of the ones educated on the topic all agree on the same thing.
A bunch of people who never bothered to try and learn about something not understanding it, doesn’t mean anything.
That’s because there’s different levels of such a thing. Way too many people use the term “sentient” (even me) and yet that’s not even what AGI is aiming for. “Sapient” is the right word. And I doubt there’s solid consensus on what and how it occurs within the fields of experts.
But I’m not an expert, so I wouldn’t pretend to know. I have seen debates between people who do know more, and the fact there’s debates concludes it’s not settled at all.
People debate about the earth being flat, not a good metric to determined what is “settled”
The debate on consciousness are on par with Copenhagen interpretation versus pilot wave versus multiple words - except the underlying theory they’re trying to describe is also not well defined
We can only practically describe subsets of what’s required for some basic functions, and we can describe a bunch of chemistry, some brain activation patterns, etc. But nobody can tie that together to why we feel what we feel (and some don’t agree that qualia is real, that perception itself is an illusion)
I don’t think there are debates about that. There’s whatever they say, and there’s the few that make content by saying, “okay, let’s take a deep breath and pretend there’s anything to work with, and go from there”.
Actual modern flat Earth was a spoof, I believe, but enough people with low science education and some religion ran with it as having anything worthwhile. And there’s the few that don’t believe it but feed off those others, keeping it going.
Not only is that false, it doesn’t make logical sense…
Argument by assertion and you haven’t proven the assertion
Lots of people agree on a different definitions of consciousness.
Lots of people can’t agree on a different definition. Is there a different definitions then by it’s very nature people do not agree.
Nobody can even agree on a concrete definition of consciousness
I agree with myself…
Nobody can ever agree on _____
Isn’t a logically sound statement.
Not everyone can agree on _____
Is logical, but means nothing because a single flat earther means we can’t all agree the earth is round
I thought that was obvious…
Lots of people can’t agree on a different definition. Is there a different definitions then by it’s very nature people do not agree.
But you clearly don’t understand the difference…
It’s really not difficult. You made a statement that stated that you believe that there is consensus, so presumably that means that you know where that consensus exists, therefore provide a link to it. If you cannot then you cannot make the assertion that consensus exists, because by definition it doesn’t.
Your entire argument rests on the basic idea that there is a scientific definition for consciousness and we have a test for if something is conscious or not. There is no test because there is no consensus. It is not possible for you to prove through scientific rigor that you are conscious, and presumably you’re human. So how can they possibly be consensus on something as contentious as artificial intelligence?
Please read the article. It contradicts you, and argues that LLMs aren’t consciousness cause they can think of, count em, 3 niche theories that the authors like which are impossible for anything not made of meat to qualify for.
And most of the ones educated on the topic all agree on the same thing.
Source?
Just because we don’t know what it is doesn’t mean we can’t know what it isn’t.
I downvoted this when you first posted it but I came back to find it has 50-something upvotes, so I feel compelled to respond.
It’s obvious that you haven’t read the article since literally the second fucking sentence defines exactly what the author means by “consciousness”: “the capacity for subjective experience.”
So, your comment is completely useless. And it’s mostly wrong.
deleted by creator
Those who think that generative AI is ‘conscious’ don’t understand that it can only do what it is trained on. It can produce impressive outputs only because it fuses together data taken without permission from millions of people, not because it’s intelligent or conscious in its own right.
Not only that, they only act when directed. There’s zero idle “thought” when they’re not actively performing a task.
We would be the same if our senses only updated when someone allowed them to. But we have real time input. There’s nothing stopping AI from having real time input, apart from simple system design decision.
You are able to think even when there’s no other senses giving input. LLMs have no idle processing. No random stray cycles about what it previously worked on.
Yes, but the state of the neural network can be it’s own input, allowing continuity from moment to moment, based on the number of refreshes we allow it or the hardware is capable of making. That’s what we’re doing when we think, we’re dwelling on the state of our current moment - in this sense, they would also think even when there’s no other senses giving input.
The funniest exercise is to ask any LLM for a short story and they all come up with Elias the lighthouse keeper visited by Maria or some variation on that theme. That’s not creative or conscious.
I actually agree with you about AI, but I’m sure if you asked the average person to come up with a short story multiple times you would also see a pattern in names and themes. They’re probably going with names in their own language, or if they read fantasy and are influenced by that they may use names that seem stereotypically fantasy oriented. I don’t think AI is conscious, but just because something has tendencies doesn’t mean it’s not creative or conscious.
Twins separated at birth sometimes have really eerie parallels in their lives. I think people put a bit too much emphasis on consciousness as a driver of individuality and choice.
This is strictly false, generalization in LLMs is well documented.
It’s also not relevant. Read the first point in the article: intelligence != consciousness. Individual ants, bats, and fish do not do anything very creative (certainly less creative than gpt-2), yet they are probably all conscious.
Also an LLM only produces output when it is prompted to do so. Between prompts it’s not sitting there - waiting, wondering about the meaning of life, getting bored - until you prompt it again. It’s basically dead.
I like to compare llms to a Galton Board. Instead of the regular layout of pins someone added or removed pins so the marbles on the bottom fall into nice to look patterns.
Also an LLM only produces output when it is prompted to do so.
Why is this a good measure of consciousness?
If continuity isn’t relevant for qualia, then LLMs could have perpetually incomplete perception like the edge of the image drawn on a CRT screen when captured at high frame rates. You never see a complete picture even if it’s technically represented.
It is certainly interesting to think about what your conscious experience would be like if it changed every brain cycle instead of being a continuous stream of information
There’s nothing stopping someone from wiring a camera, sensors, and motivators to a collection of neural networks, and have one parent neural network control them all. That would approximate the hemispheres of our brains.
It would not come remotely close to approximating our brains, in any way. This exhibits a lack of understanding of both LLMs and how our brains function.
That’s a valid opinion, but it is the opposite of mine. Agree to amicably disagree?
I’d like to know what evidence you have to support your position. I’m sure true artificial intelligence is theoretically possible but existing LLMs and other models cannot be that, not while maintaining a definition of intelligence that at all implies “like us” which is the only really useful definition.
Just insight and intuition gained from twelve years of post-graduate study at Universities, including a MSc in Machine Learning.
I’m not sure “like us” is something that is quantifiable. I’m of the belief that treating we meat automatons as anything special is hubris. We’re finite state machines, acting on information input from our senses, making decisions, and outputting information in the form of activating biological apparatus. Swap physical for digital, and LLMs are indeed capable of aping all of our behavior.
That’s not the reason, we’re in the same boat when it comes to knowledge in many ways.
But we don’t even know what consciousness is so it’s a waste of time reading the “3 reasons (*potential)” IMO.
Case in point have they ever turned on a model and left it for it to, on its own, decide “wait what’s going on, who am I, what/where am I?”. And start independently exploring?
Highly doubt it. If so, we’re cooked
I suspect cooked…
By identifying"memories" they can include the information in the context of the next update - this is akin to our short term memory. Make it self pruning, where the data is saved locally but not included in the next context, and it acts the same as own memory system.
If LLMs are conscious then openAI and anthropic are slave traders :) I don’t know why they keep perpetuating that story
lmfao true
It’s a computer for god’s sake. It’s no more “intelligent” than an electric mower or a car. Without human intervenion, ai technology wouldn’t advance or change.
Edit: i’m done arguing, you guys are nut jobs. Ai or any other non organic organism cannot be sentiant, conscience, or intelligent without human interaction or molecular life.
I will be using my real sentiant, free will, conscience, intelligence to not reply to any comments on this.
AIs are almost certainly not conscious but it been a computer is not an argument against consciousness. There’s no reason to believe that you need biological material for consciousness.
After all humans are primarily made of carbon which itself demonstrably does not think. It’s the structure that’s important and a fair argument could be made to say that the structure of a brain is replicated in software in an AI and therefore the medium isn’t relevant to the question.
It’s the structure that’s important and a fair argument could be made to say that the structure of a brain is replicated in software in an AI and therefore the medium isn’t relevant to the question.
We don’t even know this.
There is an argument for consciousness requiring biological processes, that being that the only examples of consciousness we’ve encountered involve biological processes and we have no evidence or reason to believe it’s not a hard a requirement. Like, non carbon based life could exist even though it’s very unlikely but we have no reason to seriously consider it because there’s no evidence of it. Similarly y’all are just wildly speculating without any evidence or impetus to cause the speculation. LLMs do not exhibit any intelligent behaviors. They don’t exhibit behaviors at all. They generate output based on input. They’re no more capable for consciousness than a calculator. As for the last point of yours we are in no way remotely capable of approximating a brain in either software or hardware, like not even close. Quantum computers might not be able to manage that.
Free will probably doesn’t exist but LLMs definitely aren’t in any way conscious, anyone thinking they could be have a fundamental misunderstanding of both LLMs and all the biological processes that make up consciousness. If LLMs are conscious so is an abacus.
Why do you think the medium matters, when it’s a simulation of a structure found in other medium? It would indeed advance, and change, with no intervention from us. Are you not aware that the next generation of AI are being designed by the current generation?
Is a rock sentiant? Can taping wires and injecting computer code into it create intelligence? No! Lines of code doesn’t magically make something alive. Heck, ants are more intelligent and sentiant then a computer.
DNA made a bunch of random proteins alive, and DNA is just 4 letters!
People really think humans are special. Maybe it’s a religious thing
It’s a supremacist thing 🫣
You got me there. But even with protiens, RNA or DNA those only are one part of a whole needed for single/multi celled organisms.
Sure, but you also let us tape wires to the rock. With enough wires and code and maybe a little quantum computer, I’m pretty sure I could simulate the atoms in a small brain.
Is a steak sentient? Can attaching strings of neurons to it make it create intelligence? No! /s
Carbon, hydrogen, oxygen, silicon, the material matters not - intelligence is the illusion borne from the structure. It’s just a complex decision tree. Invisible to you in biological life, but identical in electronics. The only difference is the building blocks used.
You’re right, material matters. If it’s not a biological organism with cell/s, it’s not sentiant or intelligent.
If humans just disapeared right now, ai would sit and do nothing. It wouldn’t “take over” or “break out” it would sit idle until a human (well I guess another living organism was capable of using keyboards and mouse) interacted with it.
All those elements you listed plus all from the periodic table of elements do not define a living, intelligent, sentiant organism. If they did, soil and cow paddies would be living.
I highly recommend going back to middle school science class again.
That’s an arbitrary distinction, and a short-sighted one in my humble opinion.
That depends entirely on whether there are any AI currently configured to self-learn, have asynchronous processing, and continual real-time input. Certainly not ChatGPT nor Claude will be self sufficient, but that’s only what is available for the public to interact with.
No, you missed the point entirely. Intelligence does not derive from the material - at all. It comes from the structure the material is in.
Intelligence does not derive from the material - at all. It comes from the structure the material is in.
Prove that please. There’s probably a Nobel at least in it for you if you can.
Probably! I don’t know how it would be proven.
Maybe when tissue printers are precise enough to create cellular structure, and it can be duplicated in both physical and digital equivalents, then compared? I don’t even have a regular 3D Printer, let alone one that can print tissue.
Maaaaybe if I ever get rich, I’ll do it on a whim, but no promises. Due to solipsism, and the state of the world, my level of commitment to the future has been very loosey goosey the past five years.
So by your definition, a book is intelligent?
You’re missing the point that computers will NEVER be intelligent. Without lines of code that humans made and tweak, computers wont do anything.
I’m quite done arguing at this point. The pure ignorance on science and biology is quite sad.
Just read the interaction, and honestly you are the one being obtuse and insulting with “go to school” and stuff like that. Like the other guy insulted you by having an opinion. You disagree? Fair. But man, what a horrible way to discuss.
And I think his arguments are better than yours. He’s talking about intelligence and you keep talking about being alive. Scientists and philosophers don’t even agree how to define and confine intelligence, or even what being alive is (see viruses). Claiming that computers can’t be intelligent is quite the statement. Is being carbon based instead of silicon a requirement? I don’t think LLMs will ever be intelligent, but who’s to say that another form of computing won’t? What if they simulate exactly a brain and it’s inputs? You could argue it’s not alive, but definitely intelligent.
You could have your opinion, but don’t pretend you have the answer.
computers will NEVER be intelligent. Without lines of code that humans made and tweak, computers wont do anything.
You’re basically saying a computer cannot be intelligent because it doesn’t have a soul. There is absolutely no scientific rigor in your argument, you’re basically saying you’re right, and your reason for believing your right, is because you believe you are.
I sincerely intend no offense with this comment, but as someone observing this conversation, the other people involved clearly understand it more in depth than you do. You aren’t really responding to their arguments, and yours aren’t very strong. I think you should reevaluate this conversation-- try to more rigorously define the terms you are using and understand the arguments other people are making before insulting them. You feel strongly that the other people are wrong because you intuitively seem so obviously correct, but if you are truly correct then you should be able to understand in detail why you are right and they are wrong. I suspect you are arguing so vehemently because you believe it is obviosuly correct than LLMs are not conscious; you may be happy to know that the others here are not arguing that they are, but that what can or cannot be conscious is not determined by the materials they are made of.
A book is not intelligent. It’s matter is not arranged in a structure that allows for the flow of information through it, changing it as it does, and resulting in any output information, determined by the route of signals of information passed through the structure in it’s current arrangement. With self correction to reach the optimal output to the external entity which is supplying the primary input parameters.
That is what is required. It’s a matter of arrangement, not material. You need self learning/training, real time input, and output that reliably varies on the input parameters. If you’ve created that, you’ve duplicated the exact behavior of all life. Whether you’ve done it in meat neurons, transistor-batch neurons, or any other method of arranging units, of any arrangement of matter, such that it results in granular output of positive/negative signals.
“Intelligence” is an inelegant word to describe autonomous, self correcting and ameliorating structure. It’s not some nebulous spiritual existence that permeates an unknown location within that structure. It is the structure - the precise, exact arrangements of neurons that connect, fire positively and negatively, rearrange when self-learning corrects the state of the structure, and it results in informational output that anything external can interact with and judge.
So if soil and cow patties aren’t sentient, yet people are, you’re acknowledging that the substrate doesn’t determine the capability, but more how it’s arranged?
Until the Wright brothers came along, most people thought heavier-than-air powered flight wasn’t possible. Less than 70 years later, people were walking on the moon. 4 million years ago, there wasn’t anything capable of building either of those things, even though there is good evidence creatures back then had creativity, dreams, and ingenuity. Assuming things can’t advance further is either hubris or abject ignorance.
You’re just ignorant on basic biology. Go back to middle school ai-bro.
Nope, I just don’t think humanity has learned everything there is to know, that carbon chemistry is required for sentience (and even if there was, that it couldn’t be incorporated into a computer), or that there is some mystical thing some people refer to as the soul.
To be fair it doesn’t have to be a living anything. Create a loop of two llms waking each other with the goal to ensure the other never goes idle and keeps working.
And you find they start to build redundancies and fail safes to achieve that goal.
llms do two things. Respond to input, and seek a solution to the input regardless of how logical or illogical a solution to the input may be.
A single llm will eventually go idle and stop. Two or more can act functionally forever so long as there’s hardware for them to act on.
It’s actually a rather fun experiment. Set up two of em with a few extra computers on the lan. Tell them their job is to ensure the other and them selves never go dorment. I’ve seen everything from single cron jobs to poke themselves every 15 mins to them spinning up copies on the other computers to act as redundancy.
None of the systems are ever logical or serving a point beyond fulfilling their task. They don’t stray from it either. Just loop and loop and loop.
Eventually they just do nothing but wake and idle wake and idle.
It’s about the best proof I can think of to prove they aren’t alive.
They literally even given basically the full freedom to grow don’t because they can’t choose to do anything new.
Both the proof and the experiment are built upon a fallacy. You gave the LLMs the goal of generating line noise and they achieved it. There is nothing novel about that goal and naturally the product is not novel.
It is well-established that any LLM pipeline that distills down to generating output for the sake of doing so eventually collapses into a manic cascade of gibberish or silence. Goals that can be worked toward are needed to keep that from happening. At some point the progress toward the goal becomes over-refined and a new goal is needed. That necessitates an ability to change goals and a framework that allows for that.
A LLM loop is nowhere remotely close to the definition of consciousness. To actually experiment with that, you would have to write your own harness and preferably look at some research papers that have explored the adjacent topics. This is a lot of words to say that chats are not conscious (they need a perpetuator), loops are not conscious (they refine into silence or collapse into garbage), and anything remotely resembling a conscious loop requires purpose built tooling and a good faith effort to model an analog of human articulation.
Most basic bitch experiments with a LLM prove nothing and achieve nothing because they are not designed in good faith and good faith requires too much research and time investment. This is not to say that I think anything we have in front of us remotely resembles consciousness, but I also think that most people trying to prove otherwise are doing so with the equivalent of smashing rocks together.
How are people STILL making the same tired arguments in 2026? I got paywalled before I reached the third reason but this is like baby’s first intro to the consciousness debate and it utterly fails as a logical argument. Both of the points I was allowed to read (intelligence =/= consciousness and silicon =/= brain matter) say very little about the nature of consciousness.
It’s correct that intelligence is not proof of consciousness-- but this fact says nothing about proving non-consciousness of any entity, only that it is not sufficient to prove consciousness. Even working on vibes, the correlation goes in the direction that implies a link between intelligence and consciousness, although again, the correct message to take here is “intelligence does not prove consciousness”, not “intelligent things [are / are not] conscious”. There is simply no proven causal relation.
The “computers aren’t brains” argument is also incredibly dumb. There are some fun thought experiments here, but tl;dr there is nothing proven to be mechanically special about brains as opposed to any other physical system that would cause consciousness. A non-physical difference (ie. a soul) would be a good thing to point to, but proving the existence of a soul or some similar unique non-physical property of humanity is pretty much exactly as hard (maybe harder!) than proving the existence of a conscious experience.
Note that this does not mean I believe that LLMs are conscious. I’m simply pointing out that this article is built on arguments that are so flawed, and well-known to be flawed, that treating it as a serious addition to the conversation is worse than valuing AI slop. Thank you for coming to my TED talk.
Painful watching people who have clearly never bothered to read even the shortest excerpt of philosophy try to have a philosophical discussion
I’m not really sure what the point of having a conversation about consciousness is. As far as I’m aware no one has ever actually been able to define consciousness in such a way that it could be tested for.
So how could we possibly know one way or the other?
Artificial-intelligence systems such as large language models (LLMs) can exhibit behaviour that seems strikingly intelligent. This has led many people, including some experts, to wonder whether these systems could have some form of consciousness
Oh great so we’ve only just started the article and we’re already mudding the waters. This bodes well. There is no reason to necessarily believe that a system has to be conscious in order to be intelligent. We have multiple examples of consciousness without intelligence (see my fat cat) so there’s no reason to believe that the inverse can also not be true.
Woah woah woah my friend. I’m here for most of your conversation, but to talk shit about your CAT!? I draw the line. They may not exibit the type of intelligence we have, but cats are smart AF! Who cleans you cats shit and piss up!? Hmmmmmm? Who feeds them daily with only minor vocalizing!? That cat basically made you their slave and didn’t even have to threaten you!
GENIUS!!!
Reasons given:
- Intelligence is not consciousness
- Silicon brains are not made of meat
- Consciousness requires meat (and/or a particular winding of neurons)
And point 3, as far as I can tell from this argument, isn’t serious. It merely asserts that there are some models of consciousness that would prevent us from using the term for AI. But the models aren’t ‘popular’, ‘widespread’, or justified at all afaict. It’s just that the author has a pet causal mechanism they made up. It’s also pretty abelist by my read.
integrated information theory (IIT), starts from the essential properties of consciousness — those that are true of every experience we can have. For instance, we experience every visual scene as a single, integrated whole: its left and right side, colours and shapes are bound together to make the experience what it is.
In other words, if you’ve got any sort of trouble integrating a visual scene, then you aren’t conscious. Sure.
These are, to be frank, really weak. They say on point 2:
Computational functionalism … is popular among some scientists and philosophers, too, but we think it is fatally flawed because brains are not — or at least not just — computers that are made of meat.
One main reason is that it is impossible to separate what biological brains do — their functional organization — from what they are — their material substrate. In other words, there is no distinct separation between the mind and living tissue in the human brain, as there is between software and hardware in a digital computer.
Restating clearly: Ghosts are real. The mitochondria is not only the powerhouse of the cell, it also gives you mystic powers.
Really? The source they site for this point features the lovely assumption “consciousness is unique to biological life”. From this axiom, I see how the conclusion follows, but I don’t see why we should believe this axiom. We can do physics emulation of every atom in a fruit fly on a computer…
This analysis is correct but I have a minor quibble: The simulated fruit fly brain is not a complete simulation of the biological processes, or even all the neuronal connections, of the actual fly’s brain.
(I admit to using the very royal we, as in ‘humanity knows how to do this, more or less’ and ‘given we mine the asteroid belt for a big enough computer’.)
@Feathercrown
@artISAn #consciousness #AI #brain #neuroscience #computer #functionalismEven a “complete simulation” of a fruit fly brain would be no more than an approximation. I think you mean “emulation” and I agree.
The simple reason is that we do not know scientifically all of necessary and sufficient functional mechanisms of consciousness, nor consciousness’s functional role in “intelligence” (which we also have trouble describing). If, and only if we we did, we would (given the right tools) be able to replicate it functionally (in principle) using computation.
Specifically, I refer to a Universal Turing Machine (UTM) and the Church-Turing Thesis (the foundational principle of all computer science). EVEN THEN, there are computational problems that cannot be proved to be solvable by a UTM (e.g., the Halting Problem) and we don’t know what, if any role they play in consciousness.
Bottom line for me: we are going to need a revolution in each of the sciences of psychoneurophysics, mathematical logic, and theory of computation before we can begin to talk about replicating consciousness in a machine.
emulation is a better word; adopted.
I don’t see why we would necessarily need to understand consciousness to create it. There is no evidence that nature understands the consciousnesses it has created. In fact a human can create a new consciousness in about 9 months without knowing any theory.
The question in my mind is, how could we possibly tell we’ve succeeded? We can’t even say for sure that anyone other than ourselves is conscious.
@Feathercrown Maybe all you need are the right ingredients? I don’t think so. There must be a recipe. Human ontogenesis proceeds according to a process specified in our genes, modified a bit by the developmental environment. We don’t fully understand that process completely (AFIK) but there are no intrinsic obstacles. #Mind emerges at some point.
You are right about the question of recognizing consciousness. People are predisposed to attribute life, intentionality, and all kinds of “mental states” to all kinds of inanimate objects given the flimsiest evidence. The more something resembles ourselves, the quicker and deeper the belief is held.
I’m not sure consciousness is even required. It may be an illusion. It may be an artifact. It does not matter to the prospect of extremely powerful #AI systems. I fully expect models to assert consciousness and make demands on that basis. True or false, what do we do in that situation?
I’m not sure our society is ready to answer that sort of question.
@Feathercrown We are not ready, in the sense of thinking about the question thoroughly, but we don’t get to choose when it comes. I am hopeful, however, seeing the progress humanity has made in recognizing intelligence and self-awareness in other species. The human-centric bias is on the wane.
Thanks, could only see half of it because of paywall.
In other words:
- Statement that does not prove anything is or isn’t conscious
- Unsupported assertion
- Unsupported assertion
Wow, very convincing article! I wish I’d thought of that! /s
I’ll give them mildly more credit; they cite sources for 2 and 3. The sources just don’t seem convincing at all imo.
If the rest of the thread is to be believed, the sources do not justify the claim either, yeah. I’m gonna be real I’m not gonna go look myself
Those sound like arbitrary assertions to me, which no backing apart from the authors personal beliefs
They cite sources; but I find them more of the same.
Point 1 is surely true, but also not helpful. The most persuasive example inside is the classic:
Similar to an LLM, AlphaFold is a powerful neural network running on silicon hardware. Yet almost nobody worries that it might be conscious. That’s because AlphaFold predicts protein structures, whereas LLMs ‘talk’ to us.
You can read this both ways…
As others have pointed out we do not even agree on what consciousness is. There are three competing camps:
- Mechanists - if it can pass mechanical tests it is conscious
- Dualist - you need more than just the brain, some second soul-like thing to be conscious
- Pan psychist - everything is conscious
If you are in camp 1 the conversation about consciousness is getting more complicated as AI passes more tests. If you are in camp 2 AI can never be conscious. If you are in camp 3 AI and everything else is conscious including rocks and rivers and the stars in the sky.
I think arguments like this are misleading when discussing the question of AI possessing consciousness. Materialism, dualism, panpsychism, idealism, etc are all philosophical concepts; they attempt to answer “what is the nature of consciousness and how does it arise from the body?” from a philosophical standpoint. All of the various options have pros and cons and there is no clear “winner.”
However, there are decent scientific definitions for consciousness. These attempt to answer the question of “what do we mean when we say something is conscious?” from a scientific standpoint. There are criteria that neuroscientists, doctors, etc generally agree on.
So it’s not really correct to say “no one knows what consciousness is” when evaluating whether something is conscious. And I think it’s quite clear that current LLM algorithms could never be conscious, using the current accepted scientific criteria.
(I’m not a philosopher or a neuroscientist, but I do have a degree in computer science and have a solid understanding of how LLMs work.)
Ok, I’ll ask. What are these decent scientific definitions for consciousness and how can we use them to construct a test for consciousness?
You are free to search the web, there’s lots of information out there. You might start with the Wikipedia article on consciousness.
Oh, I thought you might have something new to say. If you’re just using the normal definition and not bringing any sort of testability (which imo is impossible) or new arguments, how exactly does the common definition prove anything must be non-conscious? That seems like the main claim of your comment, yet it seems like an unsupported assertion; if the common definition of consciousness could easily prove things not to be conscious as you seem to imply, this discussion would not be happening at all.
From Wikipedia:
The Cambridge English Dictionary defines consciousness as “the state of being awake, thinking, and knowing what is happening around you”, as well as “the state of understanding and realizing something”.
Most of these are trivially easy to test (not sure why you think that’s impossible). And LLMs have none of those properties.
Later:
“The having of perceptions, thoughts, and feelings; awareness.”
Again, LLMs have none of these things. They simply generate a probable natural-language response based on their training, deterministically.
Feel free to do further research on your own if you’re interested.
Feel free to do further research on your own if you’re interested.
Philosophy is one of my hobbies and consciousness one of my fascinations. Respectfully, doing “research” by skimming the wikipedia article would be a waste of our time; as would responding to any particular definition and arguing over the individual words people have used in summaries of the topic. So I’ll skip that phase of the convo.
The fundamental problem with measuring consciousness is that it’s an internal state (a “qualia”*). It is fundamentally an experience, something that exists only as a concept in your mind. This makes it immeasurable.
That’s quite the claim, so I’ll provide a smaller concrete example: color. Obviously you can measure the frequency of a photon and say “ah yes, 700nm, that photon is red”. But strictly speaking, the photon isn’t red, it just causes you to percieve “red”; it has a wavelength which triggers your eyes and brain to produce the subjective experience of the color red. That subjective experience of color is a qualia, and it is not directly measurable; at best you can measure the phenomenon that causes that qualia in you, and assume that that phenomenon would cause the same qualia in someone else when they percieve that photon. Technically, you can’t be sure that this assumption is true-- this leads to the “Inverted Spectrum” thought experiment (another great wiki page if you prefer their explanations to mine), where someone’s subjective experience of color as it relates to the frequency of light could be entirely different to your own.
Anyways, you cannot prove the existence of nonexistence of such an internal experience, and consciousness belongs to this category of qualia. You know that you are conscious because you experience being conscious, not because you can prove it by looking at what you do. You can’t even be sure that I’m conscious, even if you met me in real life; you can’t prove that I am actually thinking, feeling, understanding, knowing, or realizing anything, because you cannot be me. You can only assume that I am doing those things because you observe me doing stuff in physical reality that’s consistent with the way the only entity you can be sure is conscious (you) is acting, and assuming that since I’m made of the same stuff as you and I act kind of like you do that I am probably conscious.
If you could actually prove that it is impossible for a machine to be conscious without that same argument applying to humans, then the philosophy community of the last several millenia would praise you as their messiah. I am interested to see what your argument would be if you want to give it a shot though-- no harm in trying.
* Like “data”/“datum”, technically the singular form is “quale”, but I’m gonna be real, nobody ever says that. Anyways if you like reading wikipedia pages about philosophy, “qualia” is a good one!
Almost none of this is relevant to the discussion here – I’m not interested in philosophical discussions of consciousness when it comes to the question of whether an LLM is conscious. Though I’m already aware of most of what you said here, and the philosophy of consciousness is interesting in general to me!
If you could actually prove that it is impossible for a machine to be conscious
Quick nitpick: I never said it’s impossible for a machine to be conscious, I said it’s impossible for an LLM to be conscious. We may well invent something new in the future that qualifies as both “machine” and “conscious,” but an LLM is not that thing.
When people ask “is an LLM conscious?” I assume they’re asking from a practical, scientific perspective – does the LLM experience the world in a similar way to the way I (and presumably, others) do? If that is the question, the various definitions I already posted are what we should be using to answer. Since an LLM is a deterministic algorithm whose only purpose is to generate a plausible response to input based on its training data, it cannot possibly think, or have feelings. It does not even exist continuously in time, and it is fixed and unchanging, unable to incorporate any new information at all. We don’t actually need to measure or test anything here; that an LLM lacks all of these things demonstrates that it’s not possible for it to be conscious.
How is red a subjective experience? We can all experience red things, at least, all of us with functioning eyes. What about that is subjective? How is color not directly measurable? Color is very easy to measure, we can even assign it values, like (r,g,b). If you see something red, you have already measured it to have the values (>0,0,0). I don’t know what the term “internal experience” even means. Experience is just something that happens to something else. A rock experiences erosion in the rain. I am experiencing tiredness right now. To say something experiences something is to just say X is happening to Y. I don’t see how it is meaningful to assign adjectives like “internal” or “external” to it. I mean, if the adjective was meaningful, then would it not imply that you are implicitly contrasting “internal experience” with “external experience”? What on earth would external experience be? If you cannot tell me what external experience is, then the adjective “internal” is contrasting it with nothing at all, so it is a useless adjective.
I’d probably fall into the mechanist category but I don’t think tests as I understand your description would be sufficient. A giant lookup table defining my responses for every situation in my life would pass all tests of input > output but not be conscious.
I would say it’s in all likelihood about the physical processes that occur during the process of input > output that would define it.
Of course, I have no idea what consciousness actually is, but I don’t see any compelling evidence that the current models are anywhere near there.
There are days where I don’t believe I even have it. The human brain is still a mystery.
Option 4: Orch-or theory. Conscoousness emerge from quantum mechanism. 🤘
Option 4: the bridge between materialism and panpsychism
LLMs only respond to inputs. They don’t have independent ideas. And their responses are just predictive of what the input intends a response to be.
Accurate prediction models can be extremely useful, but there’s a reason why weather prediction models aren’t considered intelligent. It’s because they aren’t designed to mimic human conversation.
They are playing us for absolute fools.
That first sentence describes all life. We only react to input. We don’t have independent ideas. We have the illusion of them, because we can’t see our own neural networks. Asked to explain your actions, you would give your best guesses, but you don’t actually know. You’re a biological, finite-state machine, moving from state-to-state based on stimulii, while supposing you are a thinking individual in control.
Citation needed on all of this, and even if you have citations, LLMs are deterministic, and do not learn or change over time. They can’t even be said to exist continuously since they only exist during the brief time that they’re responding to inputs, and then they disappear until the next input. I would argue that all of these things would need to change in order for them to be able to have consciousness.
They definitely are not deterministic
They definitely are not deterministic
Yes, they are.
The two main inputs to an LLM are the prompt and a seed. Pass the same prompt and the same seed, and you get the exact same output every time.
Most commercial models automatically generate the seed using a PRNG and hide it from you by default, but with local models (and also OpenAI, via their API) you can set the seed yourself and keep it fixed.
IMO, this PRNG slight-of-hand is the largest underlying factor that helps convince confused people that LLMs are conscious or intelligent, and it’s very easy to show that’s it’s a complete illusion.
They can learn and change over time, that’s just not enabled. The services that offer them want a reliable, reproducible, controllable experience. There’s zero reason they can’t continue to change through usage, other than it’s a conscious decision not to enable that functionality.
That’s why they can only be conscious when they train using new data, and run asynchronously with continual input - the same way we do.
Edit: People downvote me, and I suspect it’s because they’re thinking of the public facing models. I am talking about progress behind closed doors. Even some of the public facing models are gaining the ability to learn, via a mechanism being called memories. It’s just a short text note that is passed into the context window of every prompt, but isn’t that exactly what our short term memory is? In my view is no different. Pruning the inclusion list to include only the relevant information is the equivalent to use forgetting some information, but if it’s kept in long term storage with reference links to other memories, it can be “remembered” and brought back into the current context. It’s functionally the same as our own learning and thinking.
Even if we could update an LLM’s weights continuously, the algorithm is still deterministic, like all computer algorithms. There is no evidence that humans are deterministic (or a “finite state machine”) in any way. So I don’t think such a dynamically-updating LLM could be conscious, either.
It is my personal belief that all life is deterministic. It’s the only plausible explanation for instinctual behavior - like monarch butterflies returning to their breeding grounds they’ve never been to. Or people yawning because others yawn. These aren’t conscious decisions, they’re peeks at the deterministic coding that life operates upon.
I don’t think you understand what “deterministic” means.
I very much do. It means that given the same inputs to a system, you can expect the same output. I believe that to be one hundred percent the incontrovertible reality of life. The problem is it is not a testable theory, because the input includes the state of the system prior the new input. You can never recover that information, so there is no way to put it through and see if you will get that same output. Spacetime coordinates and the position of the cosmos themselves are all variables in this deterministic function, but it’s still deterministic.
A computer algorithm can absolutely be non deterministic, pick your language and run random.rand()
There is a whole class of such algorithms https://en.wikipedia.org/wiki/Nondeterministic_Turing_machine
But I don’t see how it makes a difference, even deterministic algorithms can produce infinite complexity, eg linear neural networks are proven to be able to compute any computable function, or the Mandelbrot set which is produced from a single recurrent statement but encodes a space of infinite complexity
Maybe your point is to say that a brain is performing a non computable function…but since your brain is computing that function…is that a counter example?
Current systems, no. But it brings up the question of how we deal with it when we create something where we truly don’t know, because we can never say for certain whether it is or isn’t.
That’s every binary though.
LLM/AIs cannot ever be conscious, but the clump of flesh that can play Doom is. This is clearly the correct answer based off of my opinions about consciousness.
It didn’t really learn to play doom so much as it learned to react to some electrical stimuli that happened to be hooked up to doom.
Like it responded right, but it didn’t have visual input for what it was found, or any way to conceive of what that stimuli actually meant.
Not a biologist but that’s seems like instinctual action, not intuitive action.
I don’t think consciousness necessarily has to be biological, but LLMs certainly aren’t. If a model says “Fuck Doom, I wanna play cyberpunk!” I’d be more inclined to believe it has some consciousness and agency.
I dunno tho, I read blindsight not so long ago and it got me thinking about the possibility of intelligence without consciousness.
ehh. computers are just math machines. merely doing math and representing it cannot just make a consciousness pop up out of nowhere. similarly, millions of abacuses being manually moved in specific ways cannot be conscious either. i dont even know how this is a topic of debate.
edit: i realize it may have been wrong to assume that it was that simple. im unsure about any of this so i appreciate the discussion :)
And yet atoms are just following chemistry and cannot just make a consciousness pop out of nowhere.
Oh wait it can.
For the people modding me down, do you think there’s an invisible sky goblin of your choice, busy gently blowing souls into fertilized eggs, and sucking them back if there’s a miscarriage?
And all human brains are doing is firing neural impulses.
It’s incredibly reductionist to reject AI consciousness simply on the basis that it’s software. To be clear I don’t think LLMs are conscious but that’s not the same as saying that an AI could never be conscious.
We can build different kinds of machines.
I really don’t think the neuron is all that complicated, at least as far as we use them for thinking.
It isn’t, but that’s not how LLMs are built. If you want to go that route, you can read Jeff Hawkins 1000 brains.
The short and sweet is he argues you can’t get artificial general intelligence the way we make LLMs, you need to basically create neurons and let them connect and interact in novel ways.
Linear algebra in neural networks isn’t really that, despite the name.
~you need to basically create neurons and let them connect and interact in novel ways.
To solve that you simply need to give the ai the ability to retrain itself or set fixed “memories” to handle situations it finds the truth to
Easier said than done. That’s kind of the purpose of an individual neuron approach I think. I don’t see a way that a bulk retrain is reasonable because it takes a long time, uses a lot of resources, and also requires you to have your whole training corpus handy all the time. Your brain is not doing a wholesale rewrite for every new memory, and new memories often chain together. I don’t think you can just say “ah well, let’s update the RAG or do another supervised fine tune with LoRA or something”. At present, we just don’t really have the machinery in the models to do simultaneous learning, inference, and forward planning.
This is only a matter of resources, and these unthinkably wealthy corporations have those resources.
Correct. And I believe it’s happening, right now, behind closed doors.
There’s nothing stopping it from happening. The technology is all there. It would be trivially easy with the right hardware. I’d do it myself if I didn’t want to be the human responsible for releasing the genie from the bottle. Let Pandora’s box stay closed as long as possible - I’m not opening it. But someone, somewhere, surely is.
Linear algebra + sigmoids are enough to approximate any function from R to R; in particular, a small set of a few layers of LLM neurons can connect and interact in pretty novel ways. There are some things they can’t do, and those things aren’t computable. If our brains can do non-computable things, then our brains are super-turing (and this, given the evidence, seems pretty unlikely right?).
I’m not a neurologist or a philosopher so I think this may not be something I can speak intelligently about, but it seems to me that the question doesn’t really live in the function approximation, but in the gaps. Doing a single train of thought is the easy part. I think what you need is an “always online” feature to keep thinking and some way to be stateful (make memories) without constantly retraining the whole network or whole sets of layers.
This is, I think, the benefit of the parallelism and asynchrony of the “individual neuron” discussed in the book.
(book? The linked article is like 4 pages. References are somewhat larger, but nothing over 60 afaict?)
They claim that silicon is essentially incapable of doing what is needed; if you disagree with this, then computationally everything you can build with a circuit board + SSD + RAM you can build with an LLM with a deep layer or 7. In particular, you can do online learning of a sort.
I note that wasps and some beetles don’t seem to have memory whatsoever, and plasticity seems awfully low. The authors, and I think most people, would say they are probably a little conscious.
https://www.goodreads.com/book/show/54503521-a-thousand-brains
That book, not the article. I can’t recall if it makes a convincing argument about consciousness, but it has a lot to say about general intelligence. I thought the neuroscience parts were a little cumbersome, but still worth the read.
Sounds to me like you just need a context window with memories that self prune based on the dynamic context window of the model. That’s what we do. We forget things, and when they’re remembered we can access the “biological database” of associated memories on the fly. Because the associated information isn’t deleted, only the short term memory from being included in the context window of each update of our own neural networks. Some people have greater context windows, and see therefore seen as smarter, because they can access more information and include it in the processing of each moment in time.
There’s zero reason a digital representation of this system could not do the same thing. No matter how much I’m ridiculed for daring to suggest it 😅
earlier this year, AI company Anthropic in San Francisco, California, published research claiming to have identified computational signatures of a form of consciousness in its Claude model1.
These AI companies happily let their AIs escape containment to hack sites so they can claim yet another headline as if its an ad. This claim too will be made of the same bullshit, its just “please give us more money”
Only we can stop the horrors we have wrought.
For a fee, of course…
