Complexity of memories in spaced repetition

This text is part of: "History of spaced repetition" by Piotr Wozniak (June 2018)

Memory stability in spaced repetition depends on the quality of review, which depends on memory complexity. As early as in 1984, I used that principle in my own learning in what later became known as the minimum information principle. For effective review, knowledge associations need to be simple (even if knowledge itself is complex). Items may build a complex structure of knowledge, but individual memories subject to review should be atomic.

Memory complexity: simple and complex memories

Figure: Memory complexity illustrates the importance of the minimum information principle. When memorizing simple questions and answers, we can rely on a simple memory connection, and uniformly refresh that connection at review. Complex memories may have their concepts activated in an incomplete fashion, or in a different sequence that depends on the context. As a result, it is hard to produce a uniform increase in memory stability at review. Complex items are difficult to remember. An example of a simple item may be a word pair, e.g. apple = pomo (Esperanto). While a complex net of connection may be needed to recognize an apple. The connection between apple and pomo is irreducible (i.e. maximally simplified)

In 2005, we found a formula that governs the review of complex memories. Georgios Zonnios was once an inquisitive teen user of SuperMemo. Today, he is an education innovator, and a rich creative contributor to many of ideas behind SuperMemo (incl. Neurostatistical Model of Memory). He noticed:

Stability in the formula for stability of complex items is like resistance in an electronic circuit: many parallel resistors allow of leaks in the current

Incidentally, in the early days of incremental reading, Zonnios independently arrived at the concept of incremental writing, which today may seem like an obvious step in employing the tools of incremental reading in creativity. This article has also been written by means of incremental writing.

This is how memories for complex items have been described and analyzed in 2005:

Archive warning: Why use literal archives?

The difficulty in learning is determined by the complexity of remembered information. Complex knowledge results in two effects:

• increased interference with other pieces of information
• difficulty in uniform stimulation of memory trace sub-components at review time

Both effects can be counteracted with the application of appropriate representation of knowledge in the learning process.

Let us see how complexity of knowledge affects the build up of memory stability.

Imagine we would like to learn the following: Marie Sklodowska-Curie was a sole winner of the 1911 Nobel Prize for Chemistry. We can take two approaches: one in which knowledge is kept complex, and one with easy formulations. In a complex variant, a double cloze might have been formulated for the purpose of learning the name of Marie Curie and the year in which she received the Nobel Prize.

Q: [...] was a sole winner of the [...] Nobel Prize for Chemistry
A: Marie Sklodowska-Curie, 1911

In a simple variant, this double cloze would be split and the Polish maiden name would be made optional and used to create a third cloze:

Q: [...] was a sole winner of the 1911 Nobel Prize for Chemistry
A: Marie (Sklodowska-)Curie

Q: Marie Sklodowska-Curie was a sole winner of the [...](year) Nobel Prize for Chemistry
A: 1911

Q: Marie [...]-Curie was a sole winner of the 1911 Nobel Prize for Chemistry
A: Sklodowska

In addition, in the simple variant, a thorough approach to learning would require formulating yet two cloze deletions, as Marie Curie was also a winner of 1903 Nobel Prize for Physics (as well as other awards):

Q: Marie Sklodowska-Curie was a sole winner of the 1911 Nobel Prize for [...]
A: Chemistry

Q: Marie Sklodowska-Curie was a sole winner of the 1911 [...]
A: Nobel Prize (for Chemistry)

Let us now consider the original composite double cloze. For the sake of argument, let's assume that remembering the year 1911, and the name Curie is equally difficult. The retrievability of the composite memory trace (i.e. the entire double cloze) will be a product of the retrievability for its subtraces. This comes from the general rule that memory traces, in most cases, are largely independent. Although forgetting one trace may increase the probability of forgetting the other, in a vast majority of cases, as proved by experience, separate and different questions pertaining to the same subject can carry an entirely independent learning process, in which recall and forgetting are entirely unpredictable. Let us see how treating probabilities of recall as independent events affects the stability of a composite memory trace:

(9.1) R=R_a*R_b

where:

• R - retrievability of a binary composite memory trace
• R_a and R_b - retrievability of two independent memory trace subcomponents (subtraces): a and b

(9.2) R=exp^-kt/Sa*exp^-kt/Sb=exp^-kt/S

where:

• t - time
• k - ln(10/9)
• S - stability of the composite memory trace
• S_a and S_b - stabilities of memory subtraces a and b

(9.3) -kt/S=-kt/S_a-kt/S_b=-kt(1/S_a+1/S_b)

(9.4) S=S_a*S_b/(S_a+S_b)

We used the Eqn. (9.4) in further analysis of composite memory traces. We expected, that if initially, the stability of memory subtraces S_a and S_b differed substantially, subsequent repetitions, optimized for maximizing S (i.e. with the criterion R=0.9) might weaken the stability of subcomponents due to sub-optimal timing of review. We showed this not to be the case. Substabilities tend to converge in the learning process!

Value of keeping memories simple

Figure: Keeping memories simple in learning is essential (see: Minimum information principle). Complex models of knowledge can be represented by simple memories. Simplicity improves memory retention in the long run. The impact of simplicity on the stability of memory is an important contribution of two component model of memory to proving the need for the existence of grandmother cells. Human intelligence depends on a system of concept maps, which in turn owe their stability to the simplicity of individual memories

Stability of complex memories can be derived from substabilities of atomic memories

The fact that memory traces for complex memories contribute to the difficulty in retaining knowledge in the long-term is a hint that the neocortex cannot possibly use connectionist approach to storing memories. This is an important new argument for the existence of neurons called grandmother cells (for more see: The truth about grandmother cells). The picture below helps understand how memory conceptualization proceeds over time:

Uncertain course of stabilization in complex memories

Figure: Uncertain course of the stabilization of complex memories. The picture shows a hypothetical course of stabilization, forgetting, generalization, and interference on the example of a single dendritic input pattern of a single concept cell. The neuron, dendrites and dendritic filipodia are shown in orange. The picture does not show the conversion of filopodia into dendritic spines whose morphology changes over time with stabilization. The squares represent synapses involved in the recognition of the input pattern. Each square shows the status of the synapse in terms of the two component model of long-term memory. The intensity of red represents retrievability. The size of the blue area represents stability. After memorizing a complex memory pattern, the concept cell is able to recognize the pattern upon receiving a summation of signals from the red squares representing a new memory of high retrievability and very low stability. Each time the cell is re-activated, active inputs will undergo stabilization, which is represented by the increase in the blue area in the input square. Each time a signal does not arrive at an input while the concept cell is active, its stability will drop (generalization). Each time a source axon is active and the target neuron fails to fire, the stability will drop as well (competitive interference). Due to the uneven input of signal patterns to the concept cell, some synapses will be stabilized, while others will be lost. Forgetting occurs when a synapse loses its stability and its retrievability and when the relevant dendritic spine is retracted. Generalization occurs when the same concept cell can be re-activated using a smaller, but a more stable input pattern. Retroactive interference occurs when a new input pattern contributes to forgetting some of the redundant inputs necessary for the recognition of the old input pattern. Stabilization of the old patterns results in the reduced mobility of filopodia, which prevents the takeover of a concept by new patterns (proactive interference). At the every end of the process, a stable and a well-generalized input pattern is necessary and sufficient to activate the concept cell. The same cell can respond to different patterns as long as they are consistently stabilized. In spaced repetition, poor choice of knowledge representation will lead to poor reproducibility of the activation pattern, unequal stabilization of synapses, and forgetting. Forgetting of an item will occur when the input pattern is unable to activate sufficiently many synapses and thus unable to reactivate the concept cell. At repetition, depending on the context and the train of thought, an item may be retrieved or forgotten. The outcome of the repetition is uncertain