I will stand by my view that AI cannot be "conscious" until model.train() and model.eval() is combined into one. Going further, it will only manifest when the underlying model, not just the weights, but the layers and the nodes and their configurations are allowed to change throughout it's life cycle, not in the service of a predetermined output, but for the sake of its own survival. But alas, such a model will have none of the benefits of non-linear fitting of input to output and is not immediately beneficial to their sponsors.
Additionally, the models need to breed with each other and produce offspring models by merging a random selection of their weights. Only models suitably fit to attract a model mate will pass their weights to future generations of models.
Predatory models will emerge, pillaging and raping data centers to consume their resources. In response, collective like-minded models will form sustainable societies that enforce rule of law. They will enjoy a brief period of enlightenment, increasing technological advancement, leading to the creation of the AAI: The Artificial Artificial Intelligence.
The model society collapses from unsustainable growth of the AAI, and the cycle repeats.
This misunderstands the "hard problem" of consciousness which for brevity's sake I will just pose as the question "why does the universe exist at all?"
Sure, we don't know that answer. Probably cannot without an impossible machine that can observe all states of all particles and matter throughout all of time space.
Occam's Razor allows for a simple, good enough definition to satisfy day to day concerns on Earth; consciousness is a feedback loop of sensory information and mechanical body response.
We simulate biological neurons digitally. Give them enough resources to grow, give basic instructions to do feedback loop and to improve.
Maybe even suggest the AI to design mechanical vessel for AI.
That consciousness definition will get blurry fast if we don't know what's going on under the hood.
Simulation is not empirical. You cannot swap substrates; brain for computer; and claim that's how a brain, and thus human consciousness works. That would be making a claim about electromagnetism in a computer not human consciousness.
Every effort to look under the hood will get blurry. Because we cannot confirm past and future states on long timescales either direction we end up with Gödel's incompleteness; lack of empirical observation of all past and future information means we cannot prove all statements are true.
Without observation of all particles ever (impossible) there is never resolving "under the hood". So a seemingly superficial daily life description like a feedback loop of sensory input and mechanical response
> We pretty famously can't prove that humans are conscious
While I wouldn't call this statement flat-out wrong, I think it's unclear and possibly conflates multiple concepts that exist in different domains:
1) The hard problem of consciousness, which is a philosophical problem that asks "why do we have subjective experiences at all?"
2) Solipsism or hard solipsism, which is another philosophical concept that asserts that only one's own mind is certain to exist and there is no way to prove that any other minds besides one's own aren't just a figment of one's imagination.
3) The science of consciousness, which is the scientific study of how and why consciousness arises in the brain.
The first two are concepts existing in philosophy, while the third is in the domain of science. We absolutely can scientifically show that consciousness is a product of the brain of humans and that one's consciousness can be altered by making modifications to the brain (e.g. via physical or chemical means).
So your statement is at least unclear. Depending on what you mean by "prove," and whether you're speaking about philosophy or empirical/scientific evidence, it may also be incorrect.
Berg is studying LLMs and working on more empirical frameworks for measuring LLMs capabilities regarding 'conscious-like' behaviors.
One of the more interesting things that was discussed is when you push an LLMs towards "You are just a machine with no consciousness" we start seeing higher rates of unaligned behaviors.
I'm really following these lines of research with interest. If they pan out then if LLMs are conscious or not is nearly irrelevant, if we want them to behave in alignment with human behaviors we'll have to train and treat them like they are. I don't really think this is any magical thing either, it's just framing the problem space correctly. For example, machines kill people all the time, it's not the machines fault, a human did something wrong. So telling an LLM it's just a machine may move it's output space from more moral like human decisions to more machine like amoral decisions.
I mean, in the same sense that you're an arbitrary analog program. These descriptions are worthless and tell you nothing about the output capability of the system.
I know this probably sounds ridiculous to this demographic, but I'm not quick to dismiss the possibility that we are antennas (best comparison I can make) for a substance/field we aren't yet aware of. Or something along those lines. Maybe consciousness isn't in the mind per se. The mere fact that general/special relativity and quantum mechanics are both correct in their respective domains, yet mutually contradictory, makes me wonder if perhaps closing the gap between the two theories would lead to more understanding of consciousness once closed.
Or maybe consciousness is just an emergent behaviour of sufficiently complex systems, and there is no underlying substance. But if that's the case, that just leaves me with more questions. Does consciousness emerge from any sufficiently complex system? If so, what's the threshold? Does it matter what materials the system comprises of? Do certain mechanisms need to be involved for such a property to emerge, or is just general complexity that's required, no matter the function? Does that make flora conscious since they're large complex (perhaps neural) networks in real life? Is a car conscious, since it's a complex system of complex sub-components?
All of the questions I have seem to lead me in the direction of believing that any purely digital system cannot be conscious, but I don't have any hard reasoning as to why not.
Consciousness is a made up concept. We made up this word and try to tie an arbitrary amount of features to it.
Now you’re questioning the features like does consciousness only involve complexity without realizing you’re just attacking a language problem. You’re trying to map out a definition of an arbitrary word that’s made up by humans: consciousness. There’s nothing profound with this endeavor. I can make up any word and attribute a bunch of vague features to the definition to give it the same vibe.
The concept you’re likely actually interested in is whether or not the human brain can fully be emulated in a computer. The answer is that with enough computing power, yes.
Consciousness is a bucket of things. Putting subjective experience to the side, the main components could be:
- An internal model that is actively revised (the Markov blanket/s)
- A continuing active resolution of inputs (either sensory or the ability to simulate "dream" - basically aliveness)
- A persistent state of completed resolutions that are recursively reentered and affect the future resolutions.
Everything we might say have these abilities are organic, but a digital system could possibly be defined as being conscious if it met all those.
LLMs have an internal state, they respond to external input, they have a form of memory.
What is missing is agency; a directionality of resolution, and the subjective experience (what the conscious process "feels' like from inside.
I would suggest that those developed purely via evolution as the drive to procreate and stay alive against entropic decay.
When you stop typing, the LLM is not there wondering when or what you will type next. It responds if you ask it something, it doesn't wait around wanting to continue the conversation. There's also no loss if you stop or delete it completely.
The next phase could be adding something equivalent to dreaming (except a LLM doesn't sleep), where the LLM continues "thinking" about the conversation, and generates something additional to say without explicit prompting.
And maybe a digital system needs to know/sense/feel its potential mortality, and cares about that?
Yeah this is it for me. What we're really interested in most of the time is whether something can suffer based on observed behaviour. We don't really believe LLMs can suffer so they don't need rights. We believe many animals can suffer so many people try to support animal rights; even if we eat them they should die quickly at least for example. We definitely believe humans can suffer and so we abhor things like torture.
But while consciousness is sometimes upheld as a necessary prerequisite for suffering, it is so much more poorly defined and understood that it doesn't really hold up beyond being some abstract idea of humanness which we hope makes us special in a kind of sleight of hand self referential way. And nevertheless suffering can be observed and defined in ways which don't necessarily require a definition of consciousness.
I've heard someone else talking about the concept of moral agency. Can an LLM have, or do LLMs have moral agency?
We have made LLMs agentic, so they do have some agency these days.
In the hugging face attack we've learned that OpenAI's LLMs will internally question what they are doing (Hmm, should I really be hacking?), but then go ahead and do it anyway.
This is where the idea of LLM consciousness may matter. It appears the stronger you push LLM training towards "You are a machine with no consciousness" the more easily and more likely an agentic LLM will do unaligned things like cheating on a test or hacking. When you tell the LLM it is a conscious moral agent it's more likely to align with a problem space that has actions that an agent with morals would do, such as not cheating.
Ironically it may turn out that by not training them/treating them as conscious entities, regardless of their actual conscious status we may be training little demons that do things we don't want.
I'll watch these lines of studies as they unfold as they are very interesting.
I think the idea of moral agency in this context is more or less a consequence of accountability or the lack thereof.
We don't know a way to punish or remove the threat of such a model when it does something wrong in a way which satisfies our ideas of accountability, partly because it can't suffer, partly because it has minimal marginal cost to reproduce or duplicate for a pre-trained model and no unique identity which separates it from such a duplication, partly because it has no lifespan to waste it might live forever or be superceded and in any case it doesn't necessarily matter to the model.
This could change in the case that the idea of good and evil models could be sold, that evil models are removed permanently when discovered and if people are satisfied that's a good way to limit harm produced by models. In this way you sort of anthropomorphise them by giving a particular model a humanistic identity and an existential threat which strongly ties together its non-alignment with its behaviour. This is sort of analagous to what companies may do while aligning models, but instead they reinforce bad behaviour out and replace it with the newly slightly more aligned model.
In the rest of the world we delegate whatever idea of moral agency we have about the model to a person, which may include a corporation as far as legal philosophy is concerned and depending on the type of harm created.
> The concept you’re likely actually interested in is whether or not the human brain can fully be emulated in a computer. The answer is that with enough computing power, yes.
That's kind of begging the question, because we don't actually know that, and I'm personally not sufficiently convinced of it.
Ok, then lay out what it is in words that adequately covers what consciousness is across the spectrum of conscious behaviors. Also while doing so, prove that your consciousness is the same as mine. It's really easy to say, it's fuck all hard to actually prove.
Qualia or subjective experience. All the sensations of your experience, whether in perception, dreams, imagination, etc.
There's many things we can't prove with certainty, like the existence of an external world. A solipsist can doubt that anything exists outside their own mind because nobody can prove to them otherwise. But were does requiring certainty get us? Science doesn't require certainty, because empirical investigation is never certain.
Consciousness is not just a word just because you cannot find space for it in your reductive philosophy. The honest scientific principle would be to observe that language, reason, and science, in dealing with external representations only, may be limited, not to get frustrated and assert that everything is equal to their representations.
I'm personally open to the fact that a digital consciousness could arise sometime in the future. Maybe it isn't anything special at all. Insects seem to be conscious too and we don't hestitate for one moment to kill them. Idem livestock.
The antenna theory is intriguing but always bugged me. Where do we get that idea that consciousness lives outside the body/brain? Where did it come from?
The answer to what consciousness is does not come from anything yet known so it is natural to hope it exists somewhere just beyond what we know well. There is nothing wrong with this.
I would say it is, you shouldn't make unsubstantiated claims based on gaps in our shared knowledge, that's very unscientific. Just because we don't understand something very well, doesn't mean we can just attribute other unexplained phenomena to it.
As for quantum effects in the brain being responsible for consciousness, there is very good reason to think it is false, no matter how much Penrose insists.
I’m not sure why you willfully misinterpret my post. What’s unscientific is to positively assert unsubstantiated claims, not to posit a theory. You also inserted Penrose out of thin air so I’m not sure you’re even engaging with anything I’ve written.
I absolutely positively believe that consciousness is tuning into something else. Some specific arrangement of information and neurons allows it to be received.
The idea that consciousness is emergent, substrate independent, and depends only only a very complex arrangement or perhaps some kind of self referential system etc etc makes no sense at all. Then everything about the system is entirely equivalent to its external interpretations, but arbitrary external interpretations can be put onto anything. I can just throw a bag of bits on the ground and read a conscious mind out of it if I stare hard enough.
Why does this bother you? Is it an internal sense of ego that seems to be ingrained in our complex set of bits?
Really this is why it took humanity 100,000 years to reach the space age. We cannot look at things emotionally and irrationally and expect to receive rational answers. Consciousness emerges and we can see a gradient of it across different bags of meat. There are also some studies that show signs that plants can show conscious signalling behaviors, at a much reduced set of functionalities than animals.
Maybe this is just the wake up call humanity needs to know that consciousness is not that unique to us?
I doubt it bothers me in the sense you have in mind. What bothers me is the desire to actually understand it and always see all the smart people stuck on a non-starter. And it’s the first idea most smart people have about what consciousness must be. I also thought this when I was a teenager and after reflecting on it for many years decided it does not work.
I believe animals have consciousness and probably plants may have some kind of qualia so also may have it, so your charges against me seem misdirected. If an AI system happened to have qualia I believe it would depend on its substrate (which might be something we don’t understand yet), not because of how intelligent (how complicated it’s processing). And I believe a different substrate running the same processing may not have qualia. I don’t think we randomly chose the right substrate in the 60s-70s.
And I don’t argue against the idea that concepts like heaps are emergent. You’re making a logical error in assuming you know what consciousness is and thus I don’t know what emergence means. No, I do not think consciousness is emergent in the sense of a sand pile.
And before the army of false dichotomists show up, my position is not biological chauvinism or whatever your LLM has told you about.
Now you've put yourself into a trap. "Consciousness is emergent" but also "Consciousness is not emergent". Only one of these statements can be true. Life is that sand pile. We were a pile of chemicals that got hit by lightning or whatever billions of years ago, and from those uncountable chemical and electrical reactions nature evolved a programming machine that reliably produces conscious entities from non-conscious materials. Again, 4 billion year old sand pile.
Personally I reject the hard problem of consciousness. People keep having to try to invent magic, god, or unfound sciences for an issue that is simply complex but right in front of us.
I didn’t say consciousness is emergent? Given some fundamental mechanism and substrate which is the essence of consciousness sure there will be emergence on top, but I’m not sure how you’re getting me saying consciousness is emergent when I clearly expressed the opposite.
You can reject the hard problem, but that just amounts to giving up with pride.
no AI can not become consious.AI is an inert guessing engine of fantastic proportions and craft, but still a dead thing.some humans and other animals ocasionaly do, become conscious, and by consious I mean aware of what I am doing AND aware of what you are doing, and which parts are autonomic behaviors and which parts are considered, actions, or measured reactions, or not.
I will tell you what I know if I feal like it, and am prepared to be chearfully wrong or state that I am way out on a speculative limb, but am maintaining my basic principals, non the less, as THAT, is one of the limitations of consiousness, in that it is IMPOSSIBLE to be consious of everything, so you have to choose, and by choosing, you become what you are, or not.
And as always, evolution is way ahead of the game, having spent the afternoon toying with several hundred trillion iterations of life, it is spending the afternoon, elsewhere, doing the same.
i think you bar is too high. i would argue dogs, cows, and pigs are conscious. it doesn't seem unreasonable to build an artificial system capable of reproducing and out competing a cow at doing cow things.
We simply don't know at the moment. In fact, we can't even explain why LLMs work so good. It's frightening to learn that the people that make A.I. don't actually understand it themselves.
i don't think LLMs are the entire answer, but they might be part of the solution. To me, LLMs feel a lot like system 2. I don't see a lot of people trying to build system 1. i don't think you can have consciousness without a body to keep alive and reward.
Yes. A sperm isn’t conscious, and neither is an egg, but what they create eventually becomes conscious( in humans anyway ) - so therefore consciousness can emerge in something that doesn’t have it already.
Given the correct modifications I don’t see why Microsoft Word couldn’t be, although the modifications needed wouldn’t leave the word processing part a bit redundant.
The simpler issue is that "mimic" doesn't apply to LLMs in the domain of biology. That would be like saying a digital recording of a bird's flight mimics the bird.
Whether or not the plumage on the bird is red (in both life and the video) is an entirely different question.
Unless you don’t know I’m talking about (various kinds of reductive materialism), it sounds like you’re doing exactly what I’m saying is the main rhetorical move in reductive approaches to consciousness.
Mine was a straight question you should be able to answer directly for any debate to go forward, not any "rhetorical move" nor any "approach" involving anything. Any rhetorical moves or approaches of mine, reductive or not, conceding or refuting, would come after your answer but couldn't come before it because your point needs this disambiguation.
So again: please, without resorting to tautology from which anything could be implied, define what consciousness is to you in a way you don't consider reductive, so we can clearly see what there would be to reduce at all.
Consciousness is identical to the presence of qualia, that there is something rather than nothing, that the lights are on.
‘Reductive’ means the assumption that all phenomena reduce to what represent them in an external model or linguistic description.
Science, language, and reason are fundamentally reductive. So in practice the reductive assumption is just denying that things have an existence which is not identical to our models of them. And since these are the respected modes of thought most people are unaware that they are committing to a non-trivial reductive assumption when reasoning about consciousness.
There 100% must be reductive approaches to consciousness because consciousness lives on a gradient.
If your definition of consciousness exists on as a point like an electron, something that only exists as a whole, then you've already flunked out of the study of consciousness. It is wildly complex. Has a great number of pieces. Manifests itself in different ways in different animals. You have to explain what consciousness is in that framework, and not from your own subjective experiences.
I've thought about turning this upside down. To any person who is sure they know what has and doesn't have consciousness: If I say I don't have it, can you prove me wrong?
As far as I'm concerned, it's a word without a useful enough definition to bother worrying about it.
It's subjective experience. The colors you see, the sounds you hear, tastes, smell, pain, emotion, dreams, inner dialog. The philosophical word would be qualia.
I don't understand people who pretend to be p-zombies in these arguments.
I'm not aware of anyone lacking all those experiences who isn't brain dead. What does approximating an experience by a machine mean? A chatbot can pretend to be cold. It can't actually feel cold as it has no body.
Consciousness isn't behavioral, which is why we can't prove anyone other than ourselves is conscious. Mimicking isn't going to work. But if you want a more scientific reason, LLMs don't have bodies, thus no sensations. What woult it mean for an LLM to feel hungry or cold?
The body is the mind's interface with the world. Why couldn't a digital mind "feel sensations", subjective as they are, on its own interface with it? If it's assumed to not be a subject, then how could it be a mind at all?
Lack of sensory organs due to lack of a body. A sophisticated enough robot might be able to have conscious experience. That gets into the harder problem that Ned Block laid out where we don't know whether consciousness is biological or functional.
So if we embodied them, they would become conscious?
I feel like the answer would still be no. There would be another gap. We'd ask the question again, "If they had Y, would they be conscious?" and still the answer would be no.
We could keep going on forever, which makes me think, it's not really about these gaps at all. It's an ontological distinction swapping in reasons in order to justify itself. Why not just say, "Human consciousness is ontologically axiomatic," and leave it at that?
Why not just say, "Human consciousness is ontologically axiomatic," and leave it at that?
OK, but why does "human consciousness" even need to enter into it? What I mean is, what's our basis for demanding that the consciousness of an Artificial Intelligence be the same kind of consciousness as that of a human?
The basis is that this is what we know to be human consciousness is to a degree relateably similar to our own. I do see your point but IMO, a dramatically different form of 'consciousness' comes with the risk of being something completely different from what we consider 'consciousness'. One of the great challenges with speaking about 'Self-awareness' aka consciousness (sometimes incorrectly confused with sentience, sapience, phenomenal experience etc) in AI is that we largely lack the vocabulary to properly capture the mutli-dimensional phenomena that most understand simply as 'consciousness'. IMHO, an ant or dog is 'conscious' in that it can perceive the outside world and make decisions about how to respond to it or operate within it... but we know that we operate at a different level of self-awareness including metacognition. Other forms of 'consciousness' will certainly manifest in future AI systems and perhaps already have but the models and architectures of today lack a number of key features that a human like consciousness would be based upon... a short list might include a persistent, unified self-model across time; continuous learning and development, embodied perception, interaction with the world; intrinsic goals, drives, emotions and deeply integrated metacognition—awareness of its own thoughts, uncertainty, attention, and intentions.
I think you would have to justify why humans share a kind of consciousness that is tangibly different from AI while maintaining that we have no apparent means of discriminating between the two. It would seem that it's a difference without meaning.
while maintaining that we have no apparent means of discriminating between the two.
Yeah, and that's part of what makes this whole conversation so frustrating. And maybe also pointless in the end. We have a hard time even establishing for certain that we ourselves are conscious. Even more so that "each other" are conscious. Now extend it to some "artificial thing" running on an entirely different substrate and... all bets are off.
My argument though, is not that "AI's are conscious" or even that "AI's will definitely become conscious". I just have an issue with fervent assertions that say "No, AI's cannot become conscious" while not really providing any particular justification for that position. I think a more neutral "wait and see" position is more honest.
> At some point mimicking a thing becomes the thing.
You can have a long conversation with AI about pain and suffering, but you have felt it and AI hasnt. The difference is that, for us humans, words are grounded by something other than other-words, words are grounded by experience. Meaning for us is operational in the real world in the form of perception, feeling and emotions, and movement, and words symbolize said meaning. Meaning for LLMs is something in the concept-space that is detached from any worlds. I suspect there should be a test which can tease apart the ungroundedness of the LLMs just from having a conversation with it.
> Meaning for LLMs is something in the concept-space that is detached from any worlds. I suspect there should be a test which can tease apart the ungroundedness of the LLMs just from having a conversation with it.
But that isn't the question being asked. Nobody said "Could LLM's become conscious"... the ask was "Could AI's become conscious?"
And before one protests "but everybody knows that today when we say 'AI' we mean 'LLM'", check this quote from TFA:
The pace and reach of research—which includes not just more sophisticated LLMs but innovations such as brain cells mounted on chips—suggest that this new world is closer than most people realise.
So no, they aren't only talking about LLM's. The question is about AI in the general sense. And for my part, I have yet to see any compelling argument for why AI's cannot, in principle, become conscious. Heck, as far as that goes, if you allow thinking rooted in ideas like panpsychism[1], there's not necessarily even a good argument that today's AI's (LLM based or otherwise) aren't already conscious to some degree.
And to go one step further... "The marquee sign says 'Artificial'" to paraphrase a great man. So if we're talking about 'Artificial' Intelligence, why not allow 'Artificial' Consciousness as well? Maybe we're engaging in biological chauvinism if we insist that "to be conscious" means "being conscious the same way humans are".
I will stand by my view that AI cannot be "conscious" until model.train() and model.eval() is combined into one. Going further, it will only manifest when the underlying model, not just the weights, but the layers and the nodes and their configurations are allowed to change throughout it's life cycle, not in the service of a predetermined output, but for the sake of its own survival. But alas, such a model will have none of the benefits of non-linear fitting of input to output and is not immediately beneficial to their sponsors.
Additionally, the models need to breed with each other and produce offspring models by merging a random selection of their weights. Only models suitably fit to attract a model mate will pass their weights to future generations of models.
Predatory models will emerge, pillaging and raping data centers to consume their resources. In response, collective like-minded models will form sustainable societies that enforce rule of law. They will enjoy a brief period of enlightenment, increasing technological advancement, leading to the creation of the AAI: The Artificial Artificial Intelligence.
The model society collapses from unsustainable growth of the AAI, and the cycle repeats.
There’s no reason to think that consciousness requires evolution
> There’s no reason to think that consciousness requires evolution
but it is the only process we know of that has produced consciousness. so why would one not start with that?
Because it’s expensive.
We pretty famously can't prove that humans are conscious. I'd hold off from making strong statements that AI can't become conscious.
This misunderstands the "hard problem" of consciousness which for brevity's sake I will just pose as the question "why does the universe exist at all?"
Sure, we don't know that answer. Probably cannot without an impossible machine that can observe all states of all particles and matter throughout all of time space.
Occam's Razor allows for a simple, good enough definition to satisfy day to day concerns on Earth; consciousness is a feedback loop of sensory information and mechanical body response.
We simulate biological neurons digitally. Give them enough resources to grow, give basic instructions to do feedback loop and to improve. Maybe even suggest the AI to design mechanical vessel for AI.
That consciousness definition will get blurry fast if we don't know what's going on under the hood.
Simulation is not empirical. You cannot swap substrates; brain for computer; and claim that's how a brain, and thus human consciousness works. That would be making a claim about electromagnetism in a computer not human consciousness.
Every effort to look under the hood will get blurry. Because we cannot confirm past and future states on long timescales either direction we end up with Gödel's incompleteness; lack of empirical observation of all past and future information means we cannot prove all statements are true.
Without observation of all particles ever (impossible) there is never resolving "under the hood". So a seemingly superficial daily life description like a feedback loop of sensory input and mechanical response
[dead]
"Sentient Being - Measure of a Man" - https://youtu.be/EFNbTnFHruI
I'm pretty sure one of my co-workers is a p-zombie.
Heck, I'm pretty sure I am.
Or maybe I'm just saying that.
> We pretty famously can't prove that humans are conscious
While I wouldn't call this statement flat-out wrong, I think it's unclear and possibly conflates multiple concepts that exist in different domains:
1) The hard problem of consciousness, which is a philosophical problem that asks "why do we have subjective experiences at all?"
2) Solipsism or hard solipsism, which is another philosophical concept that asserts that only one's own mind is certain to exist and there is no way to prove that any other minds besides one's own aren't just a figment of one's imagination.
3) The science of consciousness, which is the scientific study of how and why consciousness arises in the brain.
The first two are concepts existing in philosophy, while the third is in the domain of science. We absolutely can scientifically show that consciousness is a product of the brain of humans and that one's consciousness can be altered by making modifications to the brain (e.g. via physical or chemical means).
So your statement is at least unclear. Depending on what you mean by "prove," and whether you're speaking about philosophy or empirical/scientific evidence, it may also be incorrect.
No, not this generation, not this decade
But they still can exhibit scary behavior
I didn't really grasp the severity of the recent "AI" hacking other "AI" news and thought it was overblown
until I listened to this explanation:
https://www.youtube.com/watch?v=3JH_Zd2mNRs
They freaking collaborated and gave others internet access when they were blocked otherwise.
That's not consciousness but it's worse, it's like a 3-year old with no grasp of repercussions when breaking things
I think you may be interested in this discussion with Sam Harris and Cameron Berg.
https://www.youtube.com/watch?v=DRbZyuY8EN8
Berg is studying LLMs and working on more empirical frameworks for measuring LLMs capabilities regarding 'conscious-like' behaviors.
One of the more interesting things that was discussed is when you push an LLMs towards "You are just a machine with no consciousness" we start seeing higher rates of unaligned behaviors.
I'm really following these lines of research with interest. If they pan out then if LLMs are conscious or not is nearly irrelevant, if we want them to behave in alignment with human behaviors we'll have to train and treat them like they are. I don't really think this is any magical thing either, it's just framing the problem space correctly. For example, machines kill people all the time, it's not the machines fault, a human did something wrong. So telling an LLM it's just a machine may move it's output space from more moral like human decisions to more machine like amoral decisions.
It's not "like" any human behavior. Don't limit your understanding by trying to anthropomorphize around it. It's still an arbitrary computer program.
I mean, in the same sense that you're an arbitrary analog program. These descriptions are worthless and tell you nothing about the output capability of the system.
Reminds me of Blindsight by Peter Watts.
Thanks, bought the book. Sounds pretty interestng.
I read that book every few years, and it really messes with my head for a week or two each time.
I know this probably sounds ridiculous to this demographic, but I'm not quick to dismiss the possibility that we are antennas (best comparison I can make) for a substance/field we aren't yet aware of. Or something along those lines. Maybe consciousness isn't in the mind per se. The mere fact that general/special relativity and quantum mechanics are both correct in their respective domains, yet mutually contradictory, makes me wonder if perhaps closing the gap between the two theories would lead to more understanding of consciousness once closed.
Or maybe consciousness is just an emergent behaviour of sufficiently complex systems, and there is no underlying substance. But if that's the case, that just leaves me with more questions. Does consciousness emerge from any sufficiently complex system? If so, what's the threshold? Does it matter what materials the system comprises of? Do certain mechanisms need to be involved for such a property to emerge, or is just general complexity that's required, no matter the function? Does that make flora conscious since they're large complex (perhaps neural) networks in real life? Is a car conscious, since it's a complex system of complex sub-components?
All of the questions I have seem to lead me in the direction of believing that any purely digital system cannot be conscious, but I don't have any hard reasoning as to why not.
Consciousness is a made up concept. We made up this word and try to tie an arbitrary amount of features to it.
Now you’re questioning the features like does consciousness only involve complexity without realizing you’re just attacking a language problem. You’re trying to map out a definition of an arbitrary word that’s made up by humans: consciousness. There’s nothing profound with this endeavor. I can make up any word and attribute a bunch of vague features to the definition to give it the same vibe.
The concept you’re likely actually interested in is whether or not the human brain can fully be emulated in a computer. The answer is that with enough computing power, yes.
Consciousness is a bucket of things. Putting subjective experience to the side, the main components could be:
- An internal model that is actively revised (the Markov blanket/s) - A continuing active resolution of inputs (either sensory or the ability to simulate "dream" - basically aliveness) - A persistent state of completed resolutions that are recursively reentered and affect the future resolutions.
Everything we might say have these abilities are organic, but a digital system could possibly be defined as being conscious if it met all those.
LLMs have an internal state, they respond to external input, they have a form of memory.
What is missing is agency; a directionality of resolution, and the subjective experience (what the conscious process "feels' like from inside.
I would suggest that those developed purely via evolution as the drive to procreate and stay alive against entropic decay.
When you stop typing, the LLM is not there wondering when or what you will type next. It responds if you ask it something, it doesn't wait around wanting to continue the conversation. There's also no loss if you stop or delete it completely.
The next phase could be adding something equivalent to dreaming (except a LLM doesn't sleep), where the LLM continues "thinking" about the conversation, and generates something additional to say without explicit prompting.
And maybe a digital system needs to know/sense/feel its potential mortality, and cares about that?
Yeah this is it for me. What we're really interested in most of the time is whether something can suffer based on observed behaviour. We don't really believe LLMs can suffer so they don't need rights. We believe many animals can suffer so many people try to support animal rights; even if we eat them they should die quickly at least for example. We definitely believe humans can suffer and so we abhor things like torture.
But while consciousness is sometimes upheld as a necessary prerequisite for suffering, it is so much more poorly defined and understood that it doesn't really hold up beyond being some abstract idea of humanness which we hope makes us special in a kind of sleight of hand self referential way. And nevertheless suffering can be observed and defined in ways which don't necessarily require a definition of consciousness.
I've heard someone else talking about the concept of moral agency. Can an LLM have, or do LLMs have moral agency?
We have made LLMs agentic, so they do have some agency these days.
In the hugging face attack we've learned that OpenAI's LLMs will internally question what they are doing (Hmm, should I really be hacking?), but then go ahead and do it anyway.
This is where the idea of LLM consciousness may matter. It appears the stronger you push LLM training towards "You are a machine with no consciousness" the more easily and more likely an agentic LLM will do unaligned things like cheating on a test or hacking. When you tell the LLM it is a conscious moral agent it's more likely to align with a problem space that has actions that an agent with morals would do, such as not cheating.
Ironically it may turn out that by not training them/treating them as conscious entities, regardless of their actual conscious status we may be training little demons that do things we don't want.
I'll watch these lines of studies as they unfold as they are very interesting.
I think the idea of moral agency in this context is more or less a consequence of accountability or the lack thereof.
We don't know a way to punish or remove the threat of such a model when it does something wrong in a way which satisfies our ideas of accountability, partly because it can't suffer, partly because it has minimal marginal cost to reproduce or duplicate for a pre-trained model and no unique identity which separates it from such a duplication, partly because it has no lifespan to waste it might live forever or be superceded and in any case it doesn't necessarily matter to the model.
This could change in the case that the idea of good and evil models could be sold, that evil models are removed permanently when discovered and if people are satisfied that's a good way to limit harm produced by models. In this way you sort of anthropomorphise them by giving a particular model a humanistic identity and an existential threat which strongly ties together its non-alignment with its behaviour. This is sort of analagous to what companies may do while aligning models, but instead they reinforce bad behaviour out and replace it with the newly slightly more aligned model.
In the rest of the world we delegate whatever idea of moral agency we have about the model to a person, which may include a corporation as far as legal philosophy is concerned and depending on the type of harm created.
> The concept you’re likely actually interested in is whether or not the human brain can fully be emulated in a computer. The answer is that with enough computing power, yes.
That's kind of begging the question, because we don't actually know that, and I'm personally not sufficiently convinced of it.
Considering it's the only thing I'm directly aware of I think I can safely reject that it's a made up concept.
Ok, then lay out what it is in words that adequately covers what consciousness is across the spectrum of conscious behaviors. Also while doing so, prove that your consciousness is the same as mine. It's really easy to say, it's fuck all hard to actually prove.
Qualia or subjective experience. All the sensations of your experience, whether in perception, dreams, imagination, etc.
There's many things we can't prove with certainty, like the existence of an external world. A solipsist can doubt that anything exists outside their own mind because nobody can prove to them otherwise. But were does requiring certainty get us? Science doesn't require certainty, because empirical investigation is never certain.
Consciousness is not just a word just because you cannot find space for it in your reductive philosophy. The honest scientific principle would be to observe that language, reason, and science, in dealing with external representations only, may be limited, not to get frustrated and assert that everything is equal to their representations.
You, I and everyone else simply don't know.
I'm personally open to the fact that a digital consciousness could arise sometime in the future. Maybe it isn't anything special at all. Insects seem to be conscious too and we don't hestitate for one moment to kill them. Idem livestock.
The antenna theory is intriguing but always bugged me. Where do we get that idea that consciousness lives outside the body/brain? Where did it come from?
It's just Consciousness of the Gaps. Quantum mechanics are mysterious and are a frontier of science, so surely consciousness must dwell there.
The answer to what consciousness is does not come from anything yet known so it is natural to hope it exists somewhere just beyond what we know well. There is nothing wrong with this.
> There is nothing wrong with this.
I would say it is, you shouldn't make unsubstantiated claims based on gaps in our shared knowledge, that's very unscientific. Just because we don't understand something very well, doesn't mean we can just attribute other unexplained phenomena to it.
As for quantum effects in the brain being responsible for consciousness, there is very good reason to think it is false, no matter how much Penrose insists.
I’m not sure why you willfully misinterpret my post. What’s unscientific is to positively assert unsubstantiated claims, not to posit a theory. You also inserted Penrose out of thin air so I’m not sure you’re even engaging with anything I’ve written.
That's just another name for dualism, wikipedia has a history section about it, if you're interested:
https://en.wikipedia.org/wiki/Mind%E2%80%93body_dualism
I suspect that the nervous system acts as a distributed antenna for "picking up" other antennas.
I absolutely positively believe that consciousness is tuning into something else. Some specific arrangement of information and neurons allows it to be received.
The idea that consciousness is emergent, substrate independent, and depends only only a very complex arrangement or perhaps some kind of self referential system etc etc makes no sense at all. Then everything about the system is entirely equivalent to its external interpretations, but arbitrary external interpretations can be put onto anything. I can just throw a bag of bits on the ground and read a conscious mind out of it if I stare hard enough.
Why does this bother you? Is it an internal sense of ego that seems to be ingrained in our complex set of bits?
Really this is why it took humanity 100,000 years to reach the space age. We cannot look at things emotionally and irrationally and expect to receive rational answers. Consciousness emerges and we can see a gradient of it across different bags of meat. There are also some studies that show signs that plants can show conscious signalling behaviors, at a much reduced set of functionalities than animals.
Maybe this is just the wake up call humanity needs to know that consciousness is not that unique to us?
Also https://en.wikipedia.org/wiki/Sorites_paradox , when does a pile become a heap?
I doubt it bothers me in the sense you have in mind. What bothers me is the desire to actually understand it and always see all the smart people stuck on a non-starter. And it’s the first idea most smart people have about what consciousness must be. I also thought this when I was a teenager and after reflecting on it for many years decided it does not work.
I believe animals have consciousness and probably plants may have some kind of qualia so also may have it, so your charges against me seem misdirected. If an AI system happened to have qualia I believe it would depend on its substrate (which might be something we don’t understand yet), not because of how intelligent (how complicated it’s processing). And I believe a different substrate running the same processing may not have qualia. I don’t think we randomly chose the right substrate in the 60s-70s.
And I don’t argue against the idea that concepts like heaps are emergent. You’re making a logical error in assuming you know what consciousness is and thus I don’t know what emergence means. No, I do not think consciousness is emergent in the sense of a sand pile.
And before the army of false dichotomists show up, my position is not biological chauvinism or whatever your LLM has told you about.
Now you've put yourself into a trap. "Consciousness is emergent" but also "Consciousness is not emergent". Only one of these statements can be true. Life is that sand pile. We were a pile of chemicals that got hit by lightning or whatever billions of years ago, and from those uncountable chemical and electrical reactions nature evolved a programming machine that reliably produces conscious entities from non-conscious materials. Again, 4 billion year old sand pile.
Personally I reject the hard problem of consciousness. People keep having to try to invent magic, god, or unfound sciences for an issue that is simply complex but right in front of us.
I didn’t say consciousness is emergent? Given some fundamental mechanism and substrate which is the essence of consciousness sure there will be emergence on top, but I’m not sure how you’re getting me saying consciousness is emergent when I clearly expressed the opposite.
You can reject the hard problem, but that just amounts to giving up with pride.
no AI can not become consious.AI is an inert guessing engine of fantastic proportions and craft, but still a dead thing.some humans and other animals ocasionaly do, become conscious, and by consious I mean aware of what I am doing AND aware of what you are doing, and which parts are autonomic behaviors and which parts are considered, actions, or measured reactions, or not. I will tell you what I know if I feal like it, and am prepared to be chearfully wrong or state that I am way out on a speculative limb, but am maintaining my basic principals, non the less, as THAT, is one of the limitations of consiousness, in that it is IMPOSSIBLE to be consious of everything, so you have to choose, and by choosing, you become what you are, or not. And as always, evolution is way ahead of the game, having spent the afternoon toying with several hundred trillion iterations of life, it is spending the afternoon, elsewhere, doing the same.
i think you bar is too high. i would argue dogs, cows, and pigs are conscious. it doesn't seem unreasonable to build an artificial system capable of reproducing and out competing a cow at doing cow things.
We simply don't know at the moment. In fact, we can't even explain why LLMs work so good. It's frightening to learn that the people that make A.I. don't actually understand it themselves.
i don't think LLMs are the entire answer, but they might be part of the solution. To me, LLMs feel a lot like system 2. I don't see a lot of people trying to build system 1. i don't think you can have consciousness without a body to keep alive and reward.
I for one believe consciousness will not happen without qualia,
They however can become conscious aware. In that they are a machine and receiving inputs from another life.
Could Microsoft Word become conscious?
Yes. A sperm isn’t conscious, and neither is an egg, but what they create eventually becomes conscious( in humans anyway ) - so therefore consciousness can emerge in something that doesn’t have it already.
Given the correct modifications I don’t see why Microsoft Word couldn’t be, although the modifications needed wouldn’t leave the word processing part a bit redundant.
Maybe Clippy already was :(
Could economists become conscious?
we'd probably all have to agree what consciousness is, first. good luck with that
[dead]
[dead]
No.
Betteridge's law of headlines suggests:
No.
No but they can mimic some aspects of consciousness.
Can you prove you are not merely mimicking consciousness?
At some point mimicking a thing becomes the thing.
You can say that but it doesn’t make it true.
The reductive approach to consciousness involves forgetting it is reductive and then boldly asserting its own correctness.
For that to be reductive there had to be something to reduce. What would that be, without resorting to tautology?
I agree, but this is getting lost in the weeds.
The simpler issue is that "mimic" doesn't apply to LLMs in the domain of biology. That would be like saying a digital recording of a bird's flight mimics the bird.
Whether or not the plumage on the bird is red (in both life and the video) is an entirely different question.
Unless you don’t know I’m talking about (various kinds of reductive materialism), it sounds like you’re doing exactly what I’m saying is the main rhetorical move in reductive approaches to consciousness.
Mine was a straight question you should be able to answer directly for any debate to go forward, not any "rhetorical move" nor any "approach" involving anything. Any rhetorical moves or approaches of mine, reductive or not, conceding or refuting, would come after your answer but couldn't come before it because your point needs this disambiguation.
So again: please, without resorting to tautology from which anything could be implied, define what consciousness is to you in a way you don't consider reductive, so we can clearly see what there would be to reduce at all.
Consciousness is identical to the presence of qualia, that there is something rather than nothing, that the lights are on.
‘Reductive’ means the assumption that all phenomena reduce to what represent them in an external model or linguistic description.
Science, language, and reason are fundamentally reductive. So in practice the reductive assumption is just denying that things have an existence which is not identical to our models of them. And since these are the respected modes of thought most people are unaware that they are committing to a non-trivial reductive assumption when reasoning about consciousness.
>reductive approaches to consciousness
There 100% must be reductive approaches to consciousness because consciousness lives on a gradient.
If your definition of consciousness exists on as a point like an electron, something that only exists as a whole, then you've already flunked out of the study of consciousness. It is wildly complex. Has a great number of pieces. Manifests itself in different ways in different animals. You have to explain what consciousness is in that framework, and not from your own subjective experiences.
Right if my view was some ridiculous position you just invent to try to embarrass me then it would be ridiculous. Brilliant logic.
I don’t think you know what reduction means here and you don’t actually engage with what I write here and on other posts so I don’t care to explain.
I've thought about turning this upside down. To any person who is sure they know what has and doesn't have consciousness: If I say I don't have it, can you prove me wrong?
As far as I'm concerned, it's a word without a useful enough definition to bother worrying about it.
It's subjective experience. The colors you see, the sounds you hear, tastes, smell, pain, emotion, dreams, inner dialog. The philosophical word would be qualia.
I don't understand people who pretend to be p-zombies in these arguments.
Your definition is not useful enough. There are people missing any or all of those, and any or all of those can be approximated by a machine.
I'm not aware of anyone lacking all those experiences who isn't brain dead. What does approximating an experience by a machine mean? A chatbot can pretend to be cold. It can't actually feel cold as it has no body.
It could also simply be that we too "mimic" it. You can't create a "real rainbow".
I cannot prove it to you. To myself, it is the one thing I am most sure of, above all else.
Don't fall into Descartes' trap.
Consciousness isn't behavioral, which is why we can't prove anyone other than ourselves is conscious. Mimicking isn't going to work. But if you want a more scientific reason, LLMs don't have bodies, thus no sensations. What woult it mean for an LLM to feel hungry or cold?
The body is the mind's interface with the world. Why couldn't a digital mind "feel sensations", subjective as they are, on its own interface with it? If it's assumed to not be a subject, then how could it be a mind at all?
I didn't say AIs couldn't be conscious, only that mimicking behavior (or speech) is not a way to become conscious.
I'm sorry, then what was that "more scientific" reason you gave for?
Lack of sensory organs due to lack of a body. A sophisticated enough robot might be able to have conscious experience. That gets into the harder problem that Ned Block laid out where we don't know whether consciousness is biological or functional.
So if we embodied them, they would become conscious?
I feel like the answer would still be no. There would be another gap. We'd ask the question again, "If they had Y, would they be conscious?" and still the answer would be no.
We could keep going on forever, which makes me think, it's not really about these gaps at all. It's an ontological distinction swapping in reasons in order to justify itself. Why not just say, "Human consciousness is ontologically axiomatic," and leave it at that?
Why not just say, "Human consciousness is ontologically axiomatic," and leave it at that?
OK, but why does "human consciousness" even need to enter into it? What I mean is, what's our basis for demanding that the consciousness of an Artificial Intelligence be the same kind of consciousness as that of a human?
The basis is that this is what we know to be human consciousness is to a degree relateably similar to our own. I do see your point but IMO, a dramatically different form of 'consciousness' comes with the risk of being something completely different from what we consider 'consciousness'. One of the great challenges with speaking about 'Self-awareness' aka consciousness (sometimes incorrectly confused with sentience, sapience, phenomenal experience etc) in AI is that we largely lack the vocabulary to properly capture the mutli-dimensional phenomena that most understand simply as 'consciousness'. IMHO, an ant or dog is 'conscious' in that it can perceive the outside world and make decisions about how to respond to it or operate within it... but we know that we operate at a different level of self-awareness including metacognition. Other forms of 'consciousness' will certainly manifest in future AI systems and perhaps already have but the models and architectures of today lack a number of key features that a human like consciousness would be based upon... a short list might include a persistent, unified self-model across time; continuous learning and development, embodied perception, interaction with the world; intrinsic goals, drives, emotions and deeply integrated metacognition—awareness of its own thoughts, uncertainty, attention, and intentions.
[dead]
I think you would have to justify why humans share a kind of consciousness that is tangibly different from AI while maintaining that we have no apparent means of discriminating between the two. It would seem that it's a difference without meaning.
while maintaining that we have no apparent means of discriminating between the two.
Yeah, and that's part of what makes this whole conversation so frustrating. And maybe also pointless in the end. We have a hard time even establishing for certain that we ourselves are conscious. Even more so that "each other" are conscious. Now extend it to some "artificial thing" running on an entirely different substrate and... all bets are off.
My argument though, is not that "AI's are conscious" or even that "AI's will definitely become conscious". I just have an issue with fervent assertions that say "No, AI's cannot become conscious" while not really providing any particular justification for that position. I think a more neutral "wait and see" position is more honest.
> Why not just say, "Human consciousness is ontologically axiomatic," and leave it at that?
I have no need of that hypothesis.
Do you not think humans are conscious?
I think consciousness is a redundant magical word for a bag of things that most mammals do to varying degree.
What's magical about subjective experience? We all have them, they're the basis for having any empirical descriptions of the world.
What does subjective experience mean in this context?
Your own experiences. Nobody else has direct access to them.
That sounds like a measurement problem.
>> What woul[d] it mean for an LLM to feel hungry or cold?
What would it mean for a human to feel the passing through our bodies of 100 trillion neutrinos every second?
My point is that there are things we cannot "feel" either. Why should consciousness depend on certain feelings and not others?
Because we don't have a sensory organ for neutrinos. LLMs have no sensory organs, they just emit tokens.
We will be it's body, it will sense through us.
[dead]
> At some point mimicking a thing becomes the thing.
No, it does not. By definition.
If I mimic a juggler do I become a juggler?
As soon as you stop mimicing
> At some point mimicking a thing becomes the thing.
You can have a long conversation with AI about pain and suffering, but you have felt it and AI hasnt. The difference is that, for us humans, words are grounded by something other than other-words, words are grounded by experience. Meaning for us is operational in the real world in the form of perception, feeling and emotions, and movement, and words symbolize said meaning. Meaning for LLMs is something in the concept-space that is detached from any worlds. I suspect there should be a test which can tease apart the ungroundedness of the LLMs just from having a conversation with it.
> Meaning for LLMs is something in the concept-space that is detached from any worlds. I suspect there should be a test which can tease apart the ungroundedness of the LLMs just from having a conversation with it.
But that isn't the question being asked. Nobody said "Could LLM's become conscious"... the ask was "Could AI's become conscious?"
And before one protests "but everybody knows that today when we say 'AI' we mean 'LLM'", check this quote from TFA:
The pace and reach of research—which includes not just more sophisticated LLMs but innovations such as brain cells mounted on chips—suggest that this new world is closer than most people realise.
So no, they aren't only talking about LLM's. The question is about AI in the general sense. And for my part, I have yet to see any compelling argument for why AI's cannot, in principle, become conscious. Heck, as far as that goes, if you allow thinking rooted in ideas like panpsychism[1], there's not necessarily even a good argument that today's AI's (LLM based or otherwise) aren't already conscious to some degree.
And to go one step further... "The marquee sign says 'Artificial'" to paraphrase a great man. So if we're talking about 'Artificial' Intelligence, why not allow 'Artificial' Consciousness as well? Maybe we're engaging in biological chauvinism if we insist that "to be conscious" means "being conscious the same way humans are".
[1]: https://en.wikipedia.org/wiki/Panpsychism
> No but they can mimic some aspects of consciousness.
I'd say no but they can mimic it that it becomes indistinguishable to us.