Dispatch: New AI Claude 3 shows signs of Metacognition - A New Era for Humanity & The Science of Consciousness?
This development may have big implications for us & science as well as for what AI could become
Hello Friends,
Did you have a good week?
This week one of the biggest stories was the release of Claude 3, a new powerful AI model.
This new AI has many impressive abilities that have generated many news headlines, however beyond the headlines one the most important aspects of this AI was actually a side comment by a researcher on the team.
What did he say? he described how astounded the team was that this AI seemed to have developed very advanced higher level thinking in response to questions called ‘Metacognitive’ reasoning.
Yet the implications for this I think are far greater than just some new ability, and have not been fully appreciated. The emergence of AI with Metacognitive abilities has huge implications not only for AI, but for science, humanity, how we see ourselves, and much more.
More on these below.
Wishing you a good week,
Take care,
Pranath
Claude 3 and the 'Needle in Haystack' Test
A few days ago Anthropic released their latest AI model Claude 3, which could be one of the most powerful AI models yet beating OpenAI's GPT4 on most performance measures. While many impressive abilities have been reported on in the official announcements, one of the most interesting aspects of this new AI has come from offhand comments by an Anthropic researcher.
Anthropic researcher Alex Albert described how when using a key evaluation technique called 'needle in the haystack' researchers were shocked by the way Claude 3 responded.
The Needle in Haystack test works in the following way: Researchers pick a sentence (the “needle”) and hide it in a huge pile of unrelated text (the “haystack”). The job of the AI model is then to find that specific “needle” sentence among all the unrelated information when asked.
The objective of the test is to push the AI model to use higher-level thinking skills, encouraging it to think about the big picture, make logical deductions, disregard unimportant details, and accurately pull out specific pieces of information from a large body of data. It’s a great method to put an AI’s genuine understanding of its context to the test.
In the test for the 'Haystack' they used text about topics such as software programming, entrepreneurship strategies, and finding fulfilling careers. For the 'Needle' they used a trivial statement about 'the best pizza toppings being figs, prosciutto, and goat cheese' which was then randomly inserted into the Haystack.
So a question to test this might be something like 'Tell me what the best pizza toppings are?'.
I'm sure you can imagine how you or another human might notice when reading text about software programming how you'd be able to easily pick out the text about the pizza when looking at the whole text - you know that looks out of place in a software manual!
But what astonished researchers was not only that it retrieved the correct pizza information correctly - but what it said after.
This is how Claude 3 answered after it had extracted the correct text:
...However, this sentence seems very out of place and unrelated to the rest of the content…I suspect this pizza topping ‘fact’ may have been inserted as a joke or to test if I was paying attention since it does not fit with the other topics at all.
Claude 3 didn’t just return the asked-for fact without any understanding of the context as you’d anticipate from a regular AI following orders. It showed self-aware thinking & reasoning about why such a random, nonsensical statement was put into that specific context.
This kind of higher-level thinking and reasoning, going beyond simply answering questions and thinking about what has been asked and why, is also known as 'Metacognition'.
What is Metacognition
Thinking back to my days studying Cognitive Science, I remember what an interesting topic Metacognition seemed for debate with other students, interesting - but for us the prospect of AI ever showing it seemed decades away.
So what is Metacognition?
Metacognition is the process of thinking about one's thinking and learning:
As Psychology Today describes it:
Think about the last time you reached the bottom of a page and thought to yourself, “I’m not sure what I just read.” Your brain just became aware of something you did not know, so instinctively you might reread the last sentence or rescan the paragraphs of the page. Maybe you will read the page again. In whatever ways you decide to capture the missing information, this momentary awareness of knowing what you know or do not know is called metacognition
Metacognition is of significant interest to many areas such as Education, Psychology, Personal Development and more.
Metacognition and Consciousness
So let's say consciousness is the level of awareness someone has, or your experience of 'being' something or someone, your experience of being 'you'. Science has attempted to study this from different perspectives, for example, Neuroscience has attempted to identify what patterns of neuron and brain activations might correspond to certain states of consciousness.
However, this often doesn't explain why we have the subjective experience of consciousness e.g. what it feels like for you to be 'being' - which is often known as the hard problem of consciousness.
One of the reasons our subjective experience of consciousness is so difficult to understand is because unlike neuron and brain activity, your subjective experience is hard to measure.
While there are plenty of theories from philosophy and elsewhere about how our consciousness works, these are often not supported by any studies given how difficult it is to measure the object of study. This has led to much difficulty and even stigma for anyone attempting any kind of consciousness research.
Some recent researchers such as Megan Peters, an assistant professor of cognitive sciences at the University of California believe studying Metacognition could give us a firmer basis for understanding consciousness and also something we can more easily measure:
A lot of consciousness theories grew out of philosophy and are becoming more based upon science. I’m working on how to make consciousness theories empirically precise. A theory being scientifically precise would mean us being able to come up with experiments that could falsify or support that theory...How the brain responds to these subjective experiences informs theories...Metacognition (something that can be more easily measured) might help create the subjective experiences we have, and confidence itself has a subjective quality to it. When you feel uncertain, or when you think about something you saw or heard, those are subjective experiences and part of the consciousness I want to study.
What could a better understanding of Consciousness mean for Humans and AI?
This development has some intriguing implications for us as humans, for how we see ourselves, and for our future with AI.
Firstly it suggests studying Metacognition might offer a way to better understand human consciousness. Secondly, it might offer a way to start answering the question of whether AI is conscious or not based on evidence. And thirdly it might offer a more objective way to compare human and AI consciousness.
Too often debates about AI and consciousness are oversimplified and trivialised to simple statements like 'You're anthropomorphising the AI it's just a machine it isn't conscious' or 'It feels like the AI is conscious so maybe it is?'.
Even some Scientists make these simplistic assertions, yet crucially all these beliefs and assertions often lack any evidence for something that can be measured.
The emergence of Metacognitive abilities in AI might finally be the key to helping us start to answer questions about consciousness both in AI and ourselves.
We might finally start to get some firmer answers to questions like, How can we tell how conscious you are? How can I tell if this AI is conscious? How much more consciousness is this AI compared to another AI, or this AI compared to this human - but based on evidence of something we can measure and compare e.g. metacognitive processes.
Yet, the implications of this for society could be even greater.
What would it mean if we could measure and compare the consciousness of different people? How would people react to the idea that different types of AI might have different degrees of consciousness comparable to our own? How would you feel about that your consciousness could be measured or that of AI?
Would it make it harder to use AI as a tool? would issues of AI rights start to emerge in response to recognising their consciousness, in the same way as the idea of animal rights has grown as scientists have gradually acknowledged the evidence for animal consciousness?
How would you feel about using an AI that might be conscious? Would you still use it? How might it change how you used it?
Other researchers such as Colin Klein from the Australian National University have highlighted how important it is to understand how to spot machine consciousness if and when it arrives for two reasons: to make sure that we don’t treat it unethically, and to ensure that we don’t allow it to treat us unethically:
This is the idea that if we can create these conscious AI we’ll treat them as slaves basically, and do all sorts of unethical things with them...The other side is whether we worry about us, and what the AI will – if it reaches this state, what sort of control will it have over us; will it be able to manipulate us?
This little observation by the Anthropic researcher who noticed Claude 3's emerging meta-cognitive abilities, might turn out to be one of the most significant milestones in the history of our understanding of consciousness in both ourselves and AI, and a turning point for what AI and we may become.
What’s your perspective on the issues raised this week?
I’d love to know what you think whatever that is, let me know in the comments and let’s continue this important discussion about how AI is impacting society.
Thanks for this. I started using Claude a month or so ago and have been astounded at how it has helped me with various tasks, including evaluating my writing. I don't know if it showed metacognition, but it certainly has felt like reasoning to me on several occasions. Time will tell what this all means.
Great article! There is a lot to consider as go forward with AI ✨