Metacognition in LLMs: Foundations, Progress, and Opportunities

GitHub Repo: https://github.com/yale-nlp/LLM-Metacognition

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

• Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs (2026)
• MetaCogAgent: A Metacognitive Multi-Agent LLM Framework with Self-Aware Task Delegation (2026)
• Capability Self-Assessment: Teaching LLMs to Know Their Limits (2026)
• SCAN: A Decision-Making Framework for Effective Task Allocation with Generative AI (2026)
• Test-Time Scaling in Multimodal Foundation Models: A Comprehensive Survey of Generation and Reasoning (2026)
• A Survey of Large Language Models for Perception and Measurement of Human Psychology (2026)
• Diverse Thinking Schemata Elicit Better Reasoning in Large Language Models (2026)

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

A fascinating and comprehensive narrative survey that documents several emerging research programmes currently competing for the name “LLM metacognition." The authors are candid that no consensus definition exists, but acknowledging the messy state by saying "there is not a clear definition of what metacognition entails (for LLMs)" doesn't enlighten the simple reader if the very same paragraph then has to hold (and reconcile) several competing construals including metacognition as endogenous capacity, external monitoring or controlling function built around the model or a prompting / training technique inspired by human psychology under the same umbrella. While opting for a broad overview is a pragmatic approach, I'd wager such equivocation might actually blur the surveyed topic instead of highlighting it; the paper is not really about metacognition in LLMs, but about loosely related phenomena referred to as metacognition by someone when trying to express something LLM-typical that has no specialized vocabulary yet.

The paper takes cognitive science seriously enough, but it largely sidesteps the philosophical implications (which I tbh believe the authors somewhat wanted to tackle, since §8 explicitly raises relevant questions, but then the paper retreats to "we discuss further" or a citation each time; understandable given the genre but I felt a little blueballed there) by steering clear of the objection that these behaviors are byproducts of training data. If a LLM's confidence is just a learned distributional pattern, metrics like meta-d′ and M-ratio are measuring something detached from metacognition, though, admittedly, on any naturalist account, human confidence is also a learned pattern, produced by machinery we can't inspect and whose reports are demonstrably confabulated in part and that's precisely the reason meta-d′ was designed to be mechanism-agnostic and measure whether the system's confidence signal carries information that discriminates its own correct from incorrect responses. The question, I believe, is whether we're licensed to call that something metacognition in the case of verbalized confidence elicitation.

The neurofeedback and concept-injection part is the most exciting section of the survey, since it's the paradigm that attempts to provide causal evidence that reports track internals instead of correlational evidence that reports track accuracy. I think a taxonomy split along "functional discrimination (mechanism-agnostic, meta-d′ applies)" versus "introspective access (mechanism claims)" would have dissolved some of the definitional ambiguity (and I believe most interesting propositions will be found in the second category).

Admittedly, my understanding of metacognition is surface-level at best, so I can't really form an educated opinion on exactly what assumptions meta-d′ inherits from the equal-variance Gaussian SDT model and whether those are even coherent for prompted confidence, but I do know that debates about whether confidence reports even in humans are post-hoc constructions have a long (and contentious) history. The measurement critique, though, is pretty good and honest, as it documents that verbalized confidence values elicited in the typical 0–100 range are highly sparse and discretized, with over 75% of responses concentrating on three values, that M-ratios shift substantially depending on whether you use log-probabilities or self-reported scores, and that AUROC and M-ratio yield fully inverted model rankings.

The findings most worth remembering in my opinion (as experimental anomalies and research prompts, not settled properties of reasoning models): long reasoning traces hindering accurate self-assessment despite boosting performance, LRMs with better performance not showing the strongest metacognitive sensitivity, and the odd result that DeepSeek-R1 is less capable of metacognitive monitoring tasks such as detecting the length of its own reasoning traces while its distilled variants maintain reliable internal position estimates.

I think, actually, the main issue is the lack of vocabulary to properly express what we really mean. Trying to describe novel computational phenomena using a lexicon evolved for embodied minds drags along anthropocentric assumptions that will never cleanly transfer and are always going to depend on personal interpretation. I believe the way forward is actually not something the 300+ entries-long reference list contains, but is to bracket the folk-psychology terms entirely and develop a mechanistic, behavior-focused lexicon.

On the whole: worth reading for §4 at least, and the §5.3/§6.2 material on agentic monitoring, strategy selection, and self-improvement is also highly relevant; and it's worth reading as a (maybe somewhat methodologically premature?) bibliography too: a single reference mapping the landscape is tremendously useful, especially since it's grounded in cognitive psychology and backed by an organized reference list. At the end, I genuinely enjoyed reading the paper, so I'm not trying to be overly critical by any means. Btw, I was listening to Rindou Mikoto ASMR while reading.