Spaced repetition does not work in a classroom

From supermemo.guru
Jump to navigation Jump to search

This text is part of: "Problem of Schooling" by Piotr Wozniak (2017-2023)

Schools should not push for spaced repetition

By definition, spaced repetition is a personal tool. It cannot be used by a group. It does not belong in a classroom.

This does not prevent some great teachers asking the obvious question: Could SuperMemo or Anki improve student performance even if it violated the basic principles of spaced repetition itself? Even the visionary mind of George Zonnios tried spaced repetition in a classroom (see his conclusions). It is natural. For a teacher who cares about his students, spaced repetition is like a natural gift of better learning.

Classroom experiment

Tanagrabeast is a nick of a high school teacher who tried to share the benefits of spaced repetition in the class. It is a clear demonstration of how love and passion can make impossible happen: A year of spaced repetition in the classroom. A year later, the same author provided a more composed assessment with an honest analysis of pros and cons. The text demonstrates extensive knowledge of the theory of learning, even quoting Kahneman's description of conceptual computation. The second year of spaced repetition in the classroom leads to a conclusion that forms the basis of incremental reading:

The subconscious is a black-box back-office you can deliberately prime with study cards

Compare: Inevitability of incremental reading

Excellent classroom analysis

The message that oozes from the teacher's text is that practice easily uncovers all the truths discussed at SuperMemo Guru. This teacher could write the same articles with the same conclusions. Only the terminology would differ. There is only one stumbling block to full enlightenment. The entire analysis is made with an important assumption: "school is good, school is inevitable, and school is here to stay, so we better learn to live with it". It is clear that the mere design of school makes the whole effort incredibly ineffective. However, for a teacher full of love for his/her students, this is no obstacle. They say, love is blind.

To say that good critical analysis based on a small sample is an inferior effort is a bad school habit. The text is a goldmine of ideas but adds a bit of well-schooled self-deprecation:

Prior studies are already pretty convincing on this point and I couldn't think of a practical way to run a control group or "blind" myself

History of education

History of education is like a pendulum in which we often swing from one extreme to another without using mathematical optimization. This naturally is a larger problem that affects the entire progress of mankind. We seem to condition flawed problem solving habits at school (see: 100 bad habits learned at school). If a great deal of the population is affected, optimization is poorly understood, poorly applied, and leads to a culture of worshipping pop science in problem solving. Our teacher noticed the effect of pendulum in the approach to memory and learning

“Drill and kill” is a catchy phrase today's teachers hear in their heads whenever they use rote learning, because we've been heavily conditioned via our training and Hollywood that this is what Bad Teaching looks like. This might be one of the greatest overcorrections in the history of education. I'm not saying we should use unadorned repetition as our tool of first resort, but learning and remembering do not happen without some sort of reiteration going on, and we've made it taboo to use the most direct approach to it. We're like soldiers in a gunfight who have decided we should only kill via ricochets

The pendulum is a useless pointer in determining the value of memorization. There are pieces of knowledge that are worth preserving with spaced repetition, and pieces that are not worth the effort. The latter form a vast majority of human knowledge. The former are golden nuggets that are too precious to expose to the risk of forgetting through disuse.

Blame game of schooling

Blame game is inherent to schooling. In the presented case it necessitated a primary violation of the rules of spaced repetition. If the study had a control group, and was blinded, we could simply toss away results. The experiment had little to do with spaced repetition. It was more about teacher's effort to maximize memory in a classroom with a proxy of "group review". To me, the prime value of the experiment is to measurably reveal the ordeals of students and teachers alike. The teacher who is always given a bad hand, does not seem to blame the system. The system is a constant in the equation, this is why the teacher ultimately blames himself/herself for the outcomes:

I did not make Anki decks, assign them to my students to study independently, and then quiz them on the content. With honors classes I taught in previous years I think that might have worked, but I know my current students too well. Only about 10% of them would have done it, and the rest would have blamed me for their failing grades - with some justification, in my opinion

Luckily, schools do not entirely kill efficient learning. There is always private time after school and it can come to rescue for those who show interest:

Weekly, I updated the deck to the cloud for self-motivated students wishing to study on their own

Schools rig a blame game in which students and teachers are under endless criticism. In the end, schools take credit for what best students accomplish on their own

Students with no memory

Every teacher knows students with absolutely no memory. Those students can cram some facts for a test, and show zero recall just a week later. They can reflexively regurgitate facts or provide gamed answers in the class. They coast on autopilot passing tests and grades with little demonstrable knowledge. Spaced repetition is an excellent tool to demonstrate the futility of learning in those cases. Absence of long-term memory can be measured:

A few of my students have an impressive, empirically verified zero absorption rate. I am not exaggerating. I can compare pre-test performance to post-test performance and find absolutely no improvement, even with easy objective content that we covered extensively

School may have a negligible impact on long-term memory. The horrible inefficiency of learning at school can be measured with spaced repetition tools

Caring at will

The text provides an outline of a diagnosis. The term "caring at will" is used to describe the meta-cognitive ability to override the learn drive and provide fake valuations that feed the memory systems in the process of cramming. Naturally, this school-induced drive to learn is nothing but a bad school habit that emasculates natural semantic valuations that underlie intelligence. At school, nobody seems to care. Knowledge for the test is what counts.

If that healthy "inability to care" affects a large fraction of teenagers, we quickly understand the roots of school hate. The problem can be explained by the demands of the knowledge valuation network, individual semantic distances, and valuations. The text simplifies this neatly to "layers of indirection":

A large fraction of teenagers lack the ability to direct caring at will. They can be made to understand why they should care about something in school, but there are too many layers of indirection between it and the impulses that can actually move them .... brains in neutral can still learn. They just can't do it on purpose

Baby steps

The importance of a short semantic distance is expressed with the term "baby steps", which is worth memorizing for the sake of better elucidation of the harm of schooling:

A good card for an apathetic learner will feel unneeded on creation, because it's only a baby step away from what they already knew

It is worth noting that apathy should not be confused with learned helplessness. What looks like apathy to a teacher may simply be a reflection of strong interests that find no outlet in the classroom.

Apathy and attention

What is seen as apathy or inattention can also be associated with an inherent problem of spaced repetition. The schooled approach will ask for accumulating items and reviewing them systematically. This might be the key reason for spaced repetition failure where users swear it does not work for them. Without pleasure of learning and specific goals, repetitions became automatic reflexes of the mind. For users who make repetitions on the go, robotization of learning is worst. This is why I am not a fan of mobile SuperMemo. I tried that in the era of Pocket PC. Today, there is a simple remedy: incremental reading. Interleaving spaced repetition with new reading. Combining review with novelty.

Her name is Apathy, and she doesn't give a shit. You know how you can zone out and review something three or four times without processing it, but then you realize, kick yourself in gear, and engage your faculties, pushing it into your mind? You have to care, just a little bit. To keep it up, you have to care a little bit more. It's not always easy to find and sustain that spark of caring, because the learning target might be only indirectly instrumental to your goals

Natural creativity cycle

The harm of the disrespect for the circadian cycle and the madness of early school hours is nicely expressed as a progression from "crickets" to a "death spiral". Again, in spaced repetition, the process is measurable. All good leaners know this. All sleep researchers know it. And yet schools persist in torturing young brains and inflicting organic damage on the brain tissue:

I have one morning class where eyelids hang low, indifference runs high, and we can go several Anki cards in a row without anybody raising a hand. Crickets herald a death spiral where we can't review as many cards in a given period of time, and where don't spend as much time reviewing because the energy falls off more rapidly

Pointless curriculum

Most of teachers I know are great people and great teachers. However, there is a gaping hole in their reasoning about school and education. Due to their fish tank perspective, teachers largely do not understand the role of freedom in learning. They may understand the importance of the pleasure of learning, but they do not seem to account for the opportunity costs. If learning is good or even great, it does not mean it is optimum. Great learning with a teacher can often be substituted with greater learning that brings benefits that are larger by an order of magnitude.

In the cage of the classroom, the teacher is the boss, and she is the least likely to sense the limits on her freedom

This phenomenon leads to a form of negative selection. Once the teacher collects enough evidence on the importance of student's choice; once she realizes that the rules and regulations deprive her of her best qualities in her role, she will demand change and is likely to be fired or quit. This is what happened to legendary opponents of compulsory schooling such as John Taylor Gatto, John Holt, Danny Greenberg, Tom Durrie, and many others.

The analytical mind of our Anki teacher reveals how the seed of that awareness are planted. The rest is a matter of personality. Some teachers grit their teeth and grind on in the belief that they are more useful in the class than on the streets. The bond with the students keeps them locked in, but their conscience will never stop the whispers:

Veteran teachers start acquiring a sense of when it might be a good time to go off book and teach something that isn't in the unit, and maybe not even in the curriculum. Maybe it's teaching exactly the right word to describe a vivid situation you're reading about, or maybe it's advice on what to do in a certain type of emergency that nearly happened. As the year progressed, I found myself humoring my instincts more often because of a new confidence that I can turn an impressionable moment into a strong memory and lock it down with a new Anki card. I don't even care if it will ever be on a test. This insight has me questioning a great deal of what I thought knew about organizing a curriculum

Self-esteem

School has a knack for eroding self-esteem of nearly everyone but the very top dog in the class. A simple remedy would be to help everyone build their own strengths at their own pace in their own direction. Lowering then speed of learning or lowering the hoop to jump over can also do a lot. Spaced repetition can also become useful: it can solidify a win, and leave a permanent mark in memory. All it takes is a single memory that can serve as a seed for expanding the perception of one's own value:

It turns out that easy cards are really important because they can give wins to students who desperately need them. Knowing a 6th grade level card in a 10th grade class is no great achievement, of course, but the action takes what had been negative morale and nudges it upward. And it can trend. I can build on it. A few of these students started making Anki the thing they did in class, even if they ignored everything else. I can confidently name one student I'm sure passed my class only because of Anki. Don't get me wrong—he just barely passed. Most cards remained over his head. Anki was no miracle cure here, but it gave him and I something to work with that we didn't have when he failed my class the year before

Intelligence

The superiority of incremental reading over spaced repetition is that it builds a coherent body of abstract knowledge where rules emerge from the generalization of multiple sub-rules and facts. This generalization and crystallization process is very difficult to simulate with bare-bones spaced repetition. In incremental reading, the process of reading, reasoning, creativity, creative elaboration, and writing, and are all interleaved with the review. This orchestrates all processes involved in the natural emergence of intelligence. It is very hard to pause and pinpoint which facts or rules had a contribution to creative breakthroughs that change lives or underlie knowledge that chains into more breakthrough. The difficulty stems from the fact that forgetting, generalization, and memory optimization are all process that occur spontaneously, over time, party in sleep, partly while doing unrelated things, or while doing nothing ("idle meditation"). It would be a teacher's dream to control this process, but it is a quick path to realization it is hardly possible. There are jobs that the brain must do with no outside intervention. Even worse, intervention is often the spanner in the works. Even the best teachers will not replace the need for autonomous thinking that requires unquestioned freedom and a great deal of time that school steals from young lives:

SRS is inherently weak when it comes to the abstract and complex. No card I've devised enables a student to develop a distinctive authorial voice, or write essay openings that reveal just enough to make the reader curious. Yes, you can make cards about strategies for this sort of thing, but these were consistently my worst cards — the overly difficult "leeches" that I eventually suspended from my decks

It is a bit like with gardening. A good gardener knows when to trim, what to plant, when to nourish, and how to make the garden look great. However, when it comes to the sheer strength of the ecosystem, nothing works as well as diversity and non-intervention. No garden can compete on resilience with the rain forest.

When it comes to developing intelligence, without unconstrained freedom, even the best teacher can do more harm than good

Memorizing student names

It was wonderful to see how a teacher makes an effort to memorize all names before the year begins to provide that personal touch all kids need. Naturally, it would work better if the effort was executed incrementally overtime in proportion to getting acquainted with individual students (at least for the sake of mnemonic anchors).

Priority queue

I always thought that Anki had no priority queue, which would kill me as a user. However, our inventive teacher proved me partly wrong. Prioritizing the learning material is essential in conditions of overload. This is why a master deck may be divided into subdecks that serve as priority categories:

Due cards from subdecks higher up in the stack are shown before cards from decks listed below

School calendar

Breaks in learning evoke a spacing effect and might have benefits. However, for a spaced repetition experiment with no rescheduling and overload tools, school calendar seems to be a destroyer of the good rhythm and harmony. The psychological effect of such fits and starts must be very damaging, even for a passionate teacher, let alone apathetic student. In free learning, there is just a joy of exploration with little planning. At school, we always battle the effects of the checkered calendar, schedules and curricula. Spaced repetition exposes that ruthlessly:

We were able to recover from two weeks off in the fall readily enough, but just as we were about to catch up from the winter holidays we were beset upon by an equally lengthy spring break. Right on its heels was a short holiday week that was itself hounded by three full weeks of a special block-period testing schedule that saw each class meeting only three times a week

Terror of due material

Anki does not seem to appreciate the negative psychological impact of "due cards" (in SuperMemo, outstanding items). It might have inherited that early SuperMemo philosophy that you need to do your review daily (which is actually in part necessitated by the Algorithm SM-2). In SuperMemo for Windows, a rich set of tools makes it possible to painlessly skip a day, or a month, or a year. Once the user gets into the rhythm of learning and senses a true pleasure of learning this is rarely a problem. To know that a month-long-break will be taken care of by the algorithm is very liberating. Users of SuperMemo make a different population than users of Anki. They are more focused on the fun of learning. They are less likely to use spaced repetition for school only or for school in general. This is why breaks from learning are less epidemic. Users may go through a few cycles of dropping out, but once they root their habits in the pleasure of discovery, they become nicely systematic. Sadly, when spaced repetition comes to school, it reveals one more painful truth about schooling. School knowledge does not last. This is why schools hate vacations. The author of the article expressed similar sentiments amplified by what he could measure precisely in Anki:

Never before SRS did I fully appreciate the loss of learning that must happen every summer break

It is important to note that the loss of learning during summer breaks is actually an indictment of the entire concept of compulsory schooling. See: Summer slide

Pleasure of spacing

Algorithm SM-18 makes some users nervous due to long intervals. However, it strictly adheres to the optimum spacing marked by the forgetting index of 10% (by default). Anki uses an older algorithm that is more assuring, esp. at school. Its review is more dense. However, what is reassuring is not always good for long term learning. One of the good aspect of the spacing effect is that it adds to the perception of novelty or the sense that "I was just about to forget". This contributes to the pleasure of learning. For a teacher who works with the same decks for the second year, this can be taxing. It also points to a more general fact that routine takes away the pleasure of teaching, which ultimately hurts the student. After going though the same material in the curriculum over and over again, it is natural for a teacher to lose her passion for working with young people. That's one more painful aspect of the inefficient Prussian model of education.

It's harder to stay excited about cards you've seen dozens of times. Cracks began to show in the affable MC personality I wear for review sessions. That “apathetic third” of students I talked about last year? It's more like the apathetic supermajority when I haven't brought my 'A' game

Teachers find their passion in working with students. However, repetitiveness systematically destroys their passion for teaching the same material every year

Autonomy dilemma

A teacher faces an ancient classroom dilemma: should he rely on students doing bad job, or ask them to imitate and risk them never learning independence. The obvious answer is free learning. A student who wants to achieve a goal will learn on her own mistakes in due time. In the assembly line of school it is impossible.

[I have] a growing apprehension that I'm not doing students any great favor by writing all of their cards for them. [...] I've gained a deeper appreciation for the difficulty of creating a truly good card, so much so that I'm hesitant to trust teenagers to do it even remotely well. So the question I keep asking myself is how to give students most of the benefits of participating in the card creation process without sacrificing the time-efficiency and card quality that come from a professional writing the cards

Schools limit self-dependence in the quest for fake perfection

Decline of passions

Any great technology in a classroom risks a slow death by the loss of passion. The only effective learning is free learning. It must be powered by the learn drive. A half-baked solution in a classroom can be sustained by a teacher's enthusiasm, but it dies slowly through a feedback loop. As results are not fully satisfactory, the passion wanes, and this adversely affects the result. The example of spaced repetition is just one of a million that have been tried over the last century in classrooms around the world. This is an endless banging of a head against an immovable wall. People like John Holt, Peter Gray, Tom Durrie and others have known that for ever, but the mainstream of school system is oblivious. The use of colored cards in feedback is an example of a technology extra that adds some excitement for a while and then dies out on insufficient returns:

By December, nearly every student had stopped holding up any color other than green, perhaps tiring of the small inconvenience of considering their personal relationship with a card and rotating it to the appropriate color. Once I felt like the quality of information I was getting from them was too low to justify those few seconds I spent soliciting the feedback on each card, I stopped asking for it. This may have been a mistake. I felt like engagement levels dropped off a bit after the switch and never recovered. I don't think it had to do with feeling de-voiced, since hardly anyone was still giving actual feedback via the cards. But I think the cards might have served as a participation priming device, keeping students in the Anki mindset by giving them a way to say “Amen”

SuperMemo unnoticed

Our insightful teacher came up with a great deal of nice ideas that might improve Anki, but was apparently unaware of the fact that nearly all his ideas had been implemented in SuperMemo as far back as in 1991.

The teacher was not aware of the role of the forgetting index (1991) and the priority queue (2006) in SuperMemo. However, he clearly sensed the need for similar tools that would affect the retrievability of memory:

For some information, the most valuable thing it can be doing is bouncing around in your near-subconscious, making itself a target for collision and fusion with other ideas. […] For still other information, you want it to retrieve itself instantly as a matter of reflex before you even become consciously aware of it. […] Think grammar error recognition. Muscle memory tasks. […] Different levels of availability require different rehearsal commitments. I've not seen any explicit support for varied automaticity goals in Anki or the other spaced repetition programs I've played with

SuperMemo allows of free interval change as of Algorithm SM-11 (2002). This fact was also not known to the teacher:

I think there is room in an SRS for buttons that reduce the next interval on a card without resetting it all the way back to zero

Power of good teaching

Before anyone delves into SRS in a classroom, I might need to add a warning, and ask teachers to read "Hating SuperMemo". However, in this particular case, it seems hard to imagine that any student could possibly be put off from Anki. An utmost care was taken not to make review a burden. Perhaps some of those students will in the future get back to using spaced repetition only to discover it works much better when it is executed on one's own, in peace, and driven by goals and passions

See also

Reference


Quoted excerpts come from the following reference:

SuperMemoGuru