In this series, I am attempting to elaborate and share what the recipe of test-enhanced learning (more commonly known as retrieval practice), spaced learning, interleaving, feedback, metacognition, and motivation might look like in and out of the classroom.
I am reviewing the research and cognitive science behind these concepts and the modulators underpinning the effective retention of knowledge.
In writing this series, nine clear but interlinked elements emerged. I am considering these elements across nine distinct but related articles:
- Memory (Published: March 30, 2022)
- Repeated retrieval practice or test-enhanced learning (Published: April 6, 2022)
- Spaced retrieval practice or spaced learning (Published: April 20, 2022)
- Interleaving (Published: April 27, 2022)
- Feedback and elaboration (Published: May 4, 2022)
- Successive relearning (Published: May 11, 2022)
- Metacognition (Published: May 18, 2022)
- Overcoming illusions of competence (Published: May 25, 2022)
- Testing and motivation (this article)
I would urge readers to also listen to a recent episode of the SecEd Podcast (SecEd, 2022) looking at retrieval practice, spaced learning, and interleaving and featuring a practical discussion between myself, teacher Helen Webb, and Dr Tom Perry, who led the Education Endowment Foundation’s Cognitive Science Approaches in the Classroom review (Perry et al, 2021).
This series, in reviewing the evidence-base, seeks to help you reflect on what will work for you, your classroom, and your pupils. This is article nine and it focuses on testing and motivation.
Testing and motivation
The historic 2011 Roediger et al paper offered us “10 benefits of testing and their applications to educational practice”, citing many of the most recognised studies and researchers from the field at the point of publication. The 10 benefits were:
- The testing effect: Retrieval aids later retention.
- Testing identifies gaps in knowledge.
- Testing causes students to learn more from the next learning episode.
- Testing produces better organisation of knowledge.
- Testing improves transfer of knowledge to new contexts.
- Testing can facilitate retrieval of information that was not tested.
- Testing improves metacognitive monitoring.
- Testing prevents interference from prior material when learning new material.
- Testing provides feedback to instructors.
- Frequent testing encourages students to study.
With the exception of number eight, these benefits have all been highlighted in our series in some fashion or form. But allow me to expand and add to that 2011 list…
Testing and test expectancy
Expectation of any kind of test enhances the processing of studied material. It is an important motivator driving students to commit more effort to prepare for subsequent tests (Agarwal & Roediger, 2011; Szpunar et al, 2007; Yang et al, 2019).
The research also suggests that telling students the type of test format they should expect and the target learning goal can benefit their learning. In addition:
- Learners make more notes when they are frequently tested (Szpunar et al, 2013).
- Experiencing retrieval practice makes students “less anxious regarding upcoming tests and exams for classes in which retrieval practice was implemented” (Agarwal et al, 2014).
- A preannounced quiz encourages students to read the assigned textbook material and prepare better before class (Heiner et al, 2014).
- Frequent tests induce high test expectancy which in turn boosts test performance (Weinstein et al, 2014).
- Frequent tests reduce task-unrelated thoughts (mind wandering) while watching lecture videos (Jing et al, 2016).
- Quizzing within a course enhances not only the learning of specifically tested information, but also the learning of non-tested conceptually related information (Bjork et al, 2014).
- Testing can help reduce test anxiety and help inoculate again stress (Smith et al, 2016).
- Class quizzes increase attendance (Schrank, 2016).
- Frequent tests drive learners to allocate more time to learning (Yang et al, 2017).
It is hard not to see the value of testing.
Backward and forward effects of testing
We have discussed at length the power of the backward effect of testing – i.e. retrieval practice of previously studied information (compared to restudy). Nothing new there.
But a growing body of research is telling us that there is also a forward effect of testing – that is retrieval of previously studied information can enhance learning of subsequently presented new but conceptually related information, and also learning of information that is not necessarily related to the previously tested material.
I first encountered this when reading Szpunar et al (2008). Participants in this research studied five lists of words in anticipation of a final cumulative recall test. Prior to the experiment, participants were told to expect different activities after the presentation of each list. These activities were solving math problems, restudy of words from a “just studied” list, or immediate free recall of words from a “just studied” list.
The experimenter pretended that activities were random, but they were not. Each group had been allocated one particular activity – maths, restudy, or immediate recall testing – after studying lists one to four. However, all participants were tested immediately on the fifth list.
Two striking results emerged. Participants who had been tested immediately on lists one to four recalled about twice as many items on test five than the two non-tested groups. In addition, they showed much fewer “prior-list intrusions” than did participants in the two other groups.
So what does this tell us? These results indicate a beneficial forward effect of recall testing. This would seem to be a retrieval-specific effect which is not restricted to the learning of words but generalises to the learning of various kinds of materials.
So what is going on here?
Both encoding and retrieval explanations have been put forth to account for the forward effect of testing. These get pretty heavy, pretty quickly.
Suffice to say here that retrieval explanations suggest that recall testing between the study of a list (of information or facts) promotes “contextual list segregation”, enhancing list differentiation and reducing interference between lists. This allows participants to use context cues specific to each list and thus create more focused memory searches. I did warn you.
Encoding explanations suggest that recall testing of prior non-target materials improves encoding of the subsequently studied target material. That testing induces a “reset” of the encoding process, making the encoding of the later lists as effective as the encoding of the earlier lists.
If you are interested, there is interesting research on the forward effect of testing – also known as potentiated learning or pre-testing (Todd et al, 2021; Latimier et al, 2019; Yang et al, 2019). Indeed, Carpenter et al, 2018 state: “Studies have shown that prequestions – asking students questions before they learn something – benefit memory retention."
So interim low-stakes tests during a study phase can be used profitably to enhance the learning of new information regardless of whether it is from the same or a different domain.
Pre and interim tests induce greater test expectancy. Hence students are more likely to want to exert more effort toward encoding new information. Moreover, pre-tests induce students to exert more effort to retrieve the subsequently studied information. The recommendation is that teachers should use this knowledge to their (and their students’) advantage.
Which brings us to motivation
Motivation is not a personality trait or characteristic of a person. Kriegbaum et al (2018) reviewed the relative importance of intelligence and motivation as predictors of school achievement in a meta-analysis of 74 studies (80,145 learners).
They reported average correlations between school achievement and intelligence (0.44), motivation (0.27), and intelligence and motivation (0.17). The results show that both intelligence and motivation contribute substantially to the prediction of school achievement.
Now, considering both cognitive and motivational perspectives, and the comparative effectiveness of testing, spaced and interleaved practice, Bjork and Bjork (2020) reported “there is important research to be carried out on how motivational factors influence and interact with learning strategies”. It is hard to disagree.
Furthermore, Finn (2020) points out that students’ memories of their past academic experiences and achievements, or lack thereof, provide a basis for their expectations and goals. Such expectations and goals, in turn, can heavily influence both students’ effort to learn and their selection of learning procedures. From that standpoint, research on achievement motivation and on metacognition becomes highly relevant.
Most recently, Higham et al (2021) reported that recall of course material at the end of the semester was better for students who had taken part in repeated retrieval practice (with feedback) over multiple, spaced sessions during the semester (compared to students who had simply restudied the material).
Perhaps more importantly, the students saw further benefits including improved metacognition, increased self-reported sense of mastery, increased attentional control, and reduced anxiety. And students found successive relearning to be enjoyable and valuable.
Goldilocks
An intriguing aspect of retrieval practice is the interaction between “desirable difficulties” and pupils’ self-efficacy and sense of achievement.
By now, I hope, you will be familiar with the fact that manipulations that increase initial acquisition difficulty and enhance delayed memory performance are referred to as “desirable difficulties” (Bjork & Bjork, 2011).
Teachers and scholars have long understood that there is something of a "sweet spot" when it comes to learning. Sometimes referred to as the “Goldilocks zone”, promoting just the right amount of success to produce better learning outcomes.
When assessing students’ individual work and oral responses to class discussions, Rosenshine (2012) found that the most effective fourth-grade maths teachers had a student success rate of 82%. On the other hand, the least effective maths teachers only had success rates of 73%. Thus, his seventh principle of instruction is obtaining a high success rate – he recommends aiming for an 80% success rate, meaning that students are being challenged but not so much that they don’t succeed.
More recently, researchers at the University of Arizona reported “the 85% rule for optimal learning” (Wilson et al, 2019).
They state: “In many situations we find that there is a sweet spot in which training is neither too easy nor too hard, and where learning progresses most quickly. We find that the optimal error rate for training is around 15.87% or, conversely, that the optimal training accuracy is about 85%.
“We show theoretically that training at this optimal difficulty can lead to exponential improvements in the rate of learning.” (Wilson et al, 2019).
So, what level of difficulty is “just right”? Back to Bjork & Bjork (2020), who are very succinct in their summation in response to this question: “The level of difficulty that is optimal … will vary with the degree of a learner’s prior learning.”
Not so easy then when you have 30-plus unique learners. Is this a signpost for employing aspects of personalised learning?
Where the rubber hits the road
I have iteratively introduced and refined a spaced retrieval practice routine for learning and relearning to 21 secondary classes to date and received feedback from more than a handful of teachers. The observations and feedback are just too consistent to be ignored.
“Spaced retrieval practice-plus” promotes routined, independent, purposeful, and meaningful learning (not assessment of learning) that directs and focuses student attention. Spaced retrieval practice is demanding, requiring effortful practice, with deferred but long-term learning gains.
The “plus” is simply the use of self-assessment to leverage the gains of metacognitive monitoring and, to a lesser extent, to challenge students' metacognitive beliefs.
Forearmed with this knowledge, following a tried and tested introduction model (and potentially a metaphorical explanation to ameliorate their metacognitive beliefs), teachers consistently report a challenging introduction phase for spaced retrieval practice plus.
Predictably, after nine to 12 cycles (influenced by your subject's curriculum allocation as this affects the spacing opportunities available to you), knowledge becomes more accessible – more retrievable. Students experience tangible success. Success begets motivation. That motivation begets a greater commitment to spaced retrieval practice and learning in class. At this point, I would say that the “success-motivation-success” cycle had kicked in.
It was the consistency of the teacher reports – that spaced retrieval practice plus with high success rates, self-assessment (low-stakes, at the 8th, 9th, 10th cycle), led to a deeper commitment to learning in lesson. And so, my professional interest in the relationship between motivation and achievement increased…
What of low-stakes quizzing? As presented in article two, research is unclear. What the pupils often want to know is whether or not the quiz grade counts for anything: “When they (8th grade pupils) received a negative answer, their interest in the material dropped noticeably.” (Liming & Cuevas, 2017). Now there is a motivational conundrum to solve.
Motivation and achievement
"The impact of achievement on self-concept is greater than the impact of self-concept on achievement." This comment from a conversation with Professor Daniel Muijs, an expert in the fields of educational and teacher effectiveness, has always stuck with me.
We might also consider that “a lack of motivation is a logical response to repeated failure" – a common sentiment.
Up until recently, I held a relatively traditional perspective on motivation and achievement. Motivated students achieve and who doesn’t want to teach motivated students? Right?
What Pekrun et al (2017) attempted to do was to challenge the traditional, one-direction, correlational models on motivation, with a reciprocal (two-way) model highlighting “the importance of emotions for students’ achievement and of achievement for the development of emotions”.
Motivation directs behaviour toward achievement and therefore is known to be an important determinant of academic success. However, achievement motivation is not a single construct and subsumes such factors as motivational beliefs, task value and achievement goals.
Pekrun et al (2017) reported a "reciprocal effects model of emotion and achievement", where positive emotions (enjoyment and pride) positively predicted subsequent end-of-year maths grades, and grades in turn positively predicted the development of positive emotions.
Meanwhile, maths-related negative emotions (anger, anxiety, shame, hopelessness, boredom) were negative predictors of subsequent maths grades, and grades in turn were a predictor for the development of negative emotions.
These findings were consistent for seven discrete emotions, four time intervals, two different measures of achievement (grades, test scores), three school tracks, while controlling for students’ gender, intelligence, and critical demographic background variables.
Interestingly, Pekrun et al (2017) also suggested that these results were mediated by students’ perceptions of competence and control over achievement, again pointing to the importance of developing students' metacognitive monitoring and control capabilities (see article seven).
Remember, "emotions indeed have an influence on adolescents’ achievement, over and above the effects of general cognitive ability and prior accomplishments" (Pekrun et al, 2017).
The effects “represent a true causal influence of students’ emotion experiences” with success generally increasing perceived control, thus enhancing positive emotions, and failure expected to decrease control, leading to negative emotions (Pekrun et al, 2017).
Therefore, how do we strengthen adolescents’ positive emotions (and minimise their negative emotions)? Providing students with opportunities to “experience success” and mastery over competition goals may help to promote positive emotions and prevent negative emotions (Pekrun et al, 2014). Self-assessed, high success, spaced retrieval practice-plus would be a strong candidate.
You could always ask your students! In perhaps the most extensive study of this effect, Agarwal et al (2014) surveyed 1,408 middle and high school students who had experienced classroom-based retrieval practice programmes. The findings were that 92% of students viewed the classroom retrieval practice activities positively, believing that retrieval practice helped them learn and 72% said that frequent retrieval practice helped them feel less nervous about classroom exams.
Does the answer lie in plain sight?
According to the Expectancy Value Theory (Wigfield & Eccles, 2000), expectations of success and task value are shaped by a combination of factors.
The research has demonstrated that expectations for success and task value are distinct constructs and that expectations for success tend to predict learners' later task value. That is, learners tend to value the domains in which they feel competent. And that makes sense – right? Children are more likely to be motivated in a task if they expect to do well and if they value the activity.
We value those tasks where we have situational confidence and have experienced success in the past or are experiencing success now.
Together, expectations for success and task value have been shown to predict learner effort (motivation) and performance on learning tasks and tests.
Takeaways
- Testing is inherently good for learning. Testing improves metacognitive monitoring accuracy. Retrieval practice enhances learner confidence – all of which has clear links to motivation for learning.
- There is a forward effect of testing. Retrieval of previously studied information can enhance learning of subsequently presented new but conceptually related information.
- Pre-tests induce students to exert more effort to retrieve the subsequently studied information. Use this knowledge to your (and your students’) advantage (and pre-tests can support assessment of prior learning as well of course, identifying gaps in knowledge before a topic is tackled).
- Motivation and achievement have a reciprocal relationship.
- Work out the “success sweet-spot”. The optimal error rate could be around 15% – so plan for 85% success (Wilson et al, 2019).
- Learning is emotional and emotions influence adolescents’ achievement, over and above the effects of general cognitive ability and prior accomplishments.
- And if you only have time to read one paper on this topic: Achievement emotions and academic performance: Longitudinal models of reciprocal effects (Pekrun et al, 2017): https://bit.ly/3LgWixZ
Series conclusion
I hope you have found the nine articles in this series engaging and useful. After three years of reading and applying the research of test-enhanced learning here are some overarching takeaways.
- Be deliberate about what you teach and why you teach it.
- Testing helps identify what pupils know and do not yet know (for both teachers and pupils). Testing helps to focus your teaching.
- Testing develops metacognitive monitoring and enhances metacognition accuracy (Rivers, 2021), therefore leading to more informed study decisions.
- Repeated retrieval is key. Relearning (article six) is highly efficient and motivating. Teaching and relearning are separate learning episodes.
- Testing your pupils more at the outset accelerates and enriches learning later on. Using retrieval practice alone overlooks the benefits of pre-testing and potentiation effects (Latimier et al, 2021). Test before teaching, during teaching and as retrieval. Remember: test-enhanced learning presents an opportunity for routined, attention-demanding practice.
- Efficiency is vital: Make the best use of time, especially unsupervised study.
- Remember that the spacing or relearning may be as important, if not more important, than retrieval practice itself (Latimier et al, 2021).
- Interleaving (article four) offers a subsequent spaced relearning opportunity as well as helping learners discriminate concepts and transfer knowledge (Rohrer et al, 2019).
- Item difficulty is a factor. Difficult-to-learn information should be reviewed at shorter time intervals (Eglington et al, 2020).
- Feedback is essential – possibly more so than retrieval – and it can help the adoption of test-enhanced approaches.
- Addressing “illusions of competence” (see article eight) is an important teacher responsibility (Bjork et al, 2013). Second time around the block, the deferred benefits of test-enhanced learning are amplified…
- The most efficient schedule is a personalised one, accounting for the learner’s rates of forgetting and prior knowledge (Latimier et al, 2021).
- The importance of sleep for attention, concentration and for consolidation is unequivocal (van Dongen et al, 2012). Sleep is also a spacing opportunity.
- Kristian Still is deputy head academic at Boundary Oak School in Fareham. A school leader by day, together with his co-creator Alex Warren, a full-time senior software developer, he is also working with Leeds University and Dr Richard Allen on RememberMore, a project offering resources to teachers and pupils to support personalised spaced retrieval practice. Read his previous articles for SecEd via https://bit.ly/seced-kristianstill
References: For all research references relating to this article, go to https://bit.ly/3GXvzor
Acknowledgement: It is hard not to admire a fellow educator who can offer the Marvel anti-hero Deadpool as an analogy for test-enhanced learning. Set aside his unbounded enthusiasm for his craft and (even more so) his subject, Ben Windsor has shown that with test-enhanced learning, too. More importantly, he has a determination and faith in his students to be successful. Thanks Ben.
RememberMore: RememberMore delivers a free, personalised, and adaptive, spaced retrieval practice with feedback. For details, visit www.remembermore.app or try the app and resources via https://classroom.remembermore.app/