Metacognition: Knowing What You Do Not Know

THE LAYER THAT DIRECTS EVERY OTHER TECHNIQUE

Four posts into this series the techniques are on the table. Retrieval practice outperforms review, spacing outperforms cramming, and structured drills outperform raw repetition. Each of those techniques carries an assumption nobody states out loud: that the learner already knows which material is weak and which is solid. That assumption is where most study time gets spent badly.

The psychologist John Flavell named this layer in a 1979 paper in American Psychologist, calling it metacognition: the monitoring and regulation of one’s own cognitive processes. A student reading a chapter has a first-order process running, which is encoding. A student who pauses and asks how well that reading is actually sticking has a second-order process running on top of it. The second one decides what the first one does next, which material gets another pass, which gets abandoned, and when to stop.

Accuracy at the second level is therefore the ceiling on everything happening at the first.

JUDGMENTS OF LEARNING, AND WHY THEY DRIFT

Thomas Nelson and Louis Narens gave this monitoring layer a formal shape in 1990, splitting it into two directions of traffic. Monitoring runs bottom-up, from the object level to the meta level, and produces an assessment of how learning is going. Control runs top-down, from the meta level back to the object level, and changes what the learner does. Researchers call the specific assessment that drives most study decisions a judgment of learning: a prediction, made now, about whether a piece of material will come back later.

Those predictions are systematically wrong, and the direction of the error is consistent.

Asher Koriat and Robert Bjork published a study in 2005 documenting what they called foresight bias. When people study word pairs and immediately rate how likely they are to recall the second word given the first, their ratings run far above their actual later performance. The reason is a cue problem.

At the moment of judgment the answer is sitting right there in working memory, so the retrieval feels effortless, and the learner reads that ease as evidence of durable storage. The monitor fails the same way a fuel gauge wired to the wrong tank does: the needle moves, the reading is precise, and it describes something other than the quantity the driver needs to know.

This is the mechanism behind the illusion of fluency described in the opening post of this series. Re-reading and highlighting feel productive because they make material fluent, and fluency is the cue the monitoring system reaches for first. The technique does not build memory; it builds confidence about memory. Those are different variables, and only one of them shows up on the exam.

THE OVERCONFIDENCE TRAP

Miscalibration is not specific to studying. Baruch Fischhoff, Paul Slovic and Sarah Lichtenstein reported in 1977 that when people answered general knowledge questions and declared odds of 100 to 1 that they were correct, they were in fact wrong a substantial fraction of the time, with error rates far above what those odds imply. Extreme confidence, in their data, carried almost no extra information about accuracy.

Ever tried giving confident directions to a place you have only ever been driven to?

The best known result in this area is Justin Kruger and David Dunning’s 1999 paper in the Journal of Personality and Social Psychology, which reported that participants scoring in the bottom quartile on tests of humor, grammar and logic estimated their own performance as roughly average. The popular reading of that finding, that incompetent people are uniquely and dramatically overconfident, has not survived scrutiny well. Gilles Gignac and Marcin Zajenkowski argued in Intelligence in 2020 that most of the classic pattern is a statistical artefact produced by regression to the mean and by the specific graphical method used, and Edward Nuhfer and colleagues showed in 2016 that random numbers fed through the same analysis reproduce the signature curve. The honest version of the finding is narrower and duller than the meme: self-assessment correlates weakly with actual performance across the whole ability range, and almost everyone is somewhat overconfident.

The dull version is still the expensive one, because the cost is behavioural. John Dunlosky and Katherine Rawson tracked students across multiple study-and-test cycles in 2012 and found that overconfident learners stopped studying earlier and scored lower, and that the size of the overconfidence predicted the size of the shortfall. Poor calibration does not merely misdescribe knowledge. It terminates the process that would have produced it.

PREDICT BEFORE CHECKING

If bad monitoring causes bad control, the point of intervention sits at the monitoring step, and there is direct evidence that the link is causal rather than merely correlated. Janet Metcalfe and Bridgid Finn showed in 2008 that manipulating judgments of learning while holding actual learning constant changed which items participants chose to restudy. The judgment is not a passive readout. It is the input to the decision.

The practical move that follows is small and slightly uncomfortable: write down the prediction before looking at the answer, every time. A learner working through flashcards states, out loud or on paper, whether the next card will be recalled correctly, and only then turns it over. A programmer reading a function predicts what it returns before running it. A student closes the textbook and writes what the chapter argued before checking the summary.

The prediction converts an ordinary review into a measurement. Without it, a correct answer is just a correct answer; with it, a correct answer the learner predicted wrong becomes information about the monitoring system rather than about the material. Delaying the judgment helps as well, since Koriat and Bjork’s foresight bias shrinks substantially when the learner rates the item minutes or hours after study rather than immediately, once the answer has left working memory and the ease of retrieval stops contaminating the estimate.

CALIBRATION AS A TRAINABLE HABIT

Calibration is measurable, which means it can be practiced. The standard instrument is the Brier score, which compares stated probabilities against observed outcomes and rewards forecasts that are both confident and correct while penalising confident errors heavily. Someone who says 70 percent and is right roughly 70 percent of the time is well calibrated; someone who says 95 percent and is right 60 percent of the time is not, regardless of how often they are technically correct. The score works like a bathroom scale checked against a known weight, indifferent to how heavy the reading feels and interested only in how far it sits from the truth.

Barbara Mellers and colleagues reported in 2015 on the Good Judgment Project, a large forecasting tournament in which thousands of participants made probabilistic predictions about world events over several years. Accuracy improved with training in probabilistic reasoning, with tracking of past forecasts, and with practice, which is the relevant point here: calibration behaved like a skill, not a fixed trait.

Four habits carry most of that effect into ordinary study. Attach a number to every prediction rather than a vague feeling, because fairly confident cannot be scored and 70 percent can. Keep the predictions somewhere durable, since memory of past confidence is itself reconstructed and flattering.

Make judgments after a delay rather than immediately. And treat every practice test as an instrument reading rather than a performance, which is the point where this post rejoins active recall: the test measures the material and audits the monitor at the same time.

WHAT CALIBRATION DOES NOT FIX

Accurate self-assessment routes effort; it does not generate capability. A perfectly calibrated learner still has to do the retrieval practice, still has to space it, and still has to build the drills. Robert Bjork, John Dunlosky and Nate Kornell noted in their 2013 review of self-regulated learning that learners hold durable false beliefs about how learning works, and that better monitoring corrects the allocation of study time without touching those underlying beliefs.

Remember the fuel gauge from earlier: fixing it tells the driver how much fuel is in the tank, and nothing else. It does not add range.

Calibration is also domain-bound. Someone well calibrated about their chemistry knowledge has no particular advantage judging their own driving, their management skill, or their grasp of a language they last used a decade ago. The habit transfers; the accuracy does not.

What it buys is narrower than the enthusiasm around it suggests and more valuable than the alternative. A learner who knows the map of their own ignorance stops studying what they already know, which is the single most common way study time disappears. Whether that map can be kept current in a subject that keeps changing underneath it is a harder question, and one this series has not answered yet.

T.

References

1. Metacognition and Cognitive Monitoring: A New Area of Cognitive-Developmental Inquiry - Flavell (1979), American Psychologist. The paper that named metacognition and framed it as monitoring plus regulation of one’s own cognition.

2. Metamemory: A Theoretical Framework and New Findings - Nelson & Narens (1990), Psychology of Learning and Motivation. Establishes the object-level and meta-level split, with monitoring running bottom-up and control running top-down.

3. Illusions of Competence in Monitoring One’s Knowledge During Study - Koriat & Bjork (2005), Journal of Experimental Psychology: Learning, Memory, and Cognition. Documents foresight bias and shows that delaying the judgment reduces it.

4. Knowing With Certainty: The Appropriateness of Extreme Confidence - Fischhoff, Slovic & Lichtenstein (1977), Journal of Experimental Psychology: Human Perception and Performance. Reports how poorly extreme stated odds track actual accuracy.

5. Unskilled and Unaware of It - Kruger & Dunning (1999), Journal of Personality and Social Psychology. The original study behind the popular effect, reporting bottom-quartile participants estimating themselves near average.

6. The Dunning-Kruger Effect Is (Mostly) a Statistical Artefact - Gignac & Zajenkowski (2020), Intelligence. Argues the classic pattern largely reflects regression to the mean and the graphical method used.

7. Random Number Simulations Reveal How Random Noise Affects the Measurements and Graphical Portrayals of Self-Assessed Competency - Nuhfer, Cogan, Kloock, Wood & Gaze (2016), Numeracy. Shows random data reproducing the signature Dunning-Kruger curve.

8. Overconfidence Produces Underachievement - Dunlosky & Rawson (2012), Learning and Instruction. Links inaccurate self-evaluation to premature termination of study and lower retention.

9. Evidence That Judgments of Learning Are Causally Related to Study Choice - Metcalfe & Finn (2008), Psychonomic Bulletin & Review. Demonstrates that changing the judgment changes restudy decisions, establishing the causal direction.

10. Identifying and Cultivating Superforecasters as a Method of Improving Probabilistic Predictions - Mellers, Stone, Murray, Minster and colleagues (2015), Perspectives on Psychological Science. Good Judgment Project results showing calibration improving with training and tracking.

11. Self-Regulated Learning: Beliefs, Techniques, and Illusions - Bjork, Dunlosky & Kornell (2013), Annual Review of Psychology. Review of why learners hold persistent false beliefs about their own learning.

12. Brier Score - Overview of the scoring rule used to measure the accuracy of probabilistic predictions, and the standard instrument for tracking calibration over time.