The prescription was everywhere as the full extent of academic damage wrought by the pandemic first became clear: Set up tutoring programs to catch students up.
The same advice echoed from policy papers, think tanks, and the federal government, which put hundreds of billions into school recovery dollars. Experts and researchers heralded tutoring as an evidence-backed solution for addressing significant academic gaps, and some school systems saw early successes.
Now, though, several evaluations of pandemic-era tutoring programs are showing than expected鈥攐r revealing that .
The most recent of these, from researchers at Stanford University鈥檚 SCALE Initiative, in a large, urban district during the 2022-23 and 2023-24 school years. Neither led to overall gains in academic achievement.
But when researchers dug deeper into the data, they identified implementation problems that could be driving these null effects.
The findings add to a growing body of evidence around tutoring in a post-COVID landscape that suggests the effectiveness of a program hinges on the nitty-gritty details of how it is run鈥攈ow often students meet with their tutors, for instance, or whether lessons are tailored to their specific needs.
Studying these implementation details could help school systems build more effective tutoring initiatives in the long run, said Elizabeth Huffaker, an assistant professor of educational leadership and policy at the University of Florida, and the lead author on the SCALE paper.
鈥淭here is a lot of potential upside to findings like this, even though we wish that students were seeing more across-the-board gains as soon as possible,鈥� she said.
Successful tutoring is hard to replicate at scale
The tutoring programs that districts stood up over the past five years are still in relatively early stages, said Matthew Kraft, a professor of education and economics at Brown University who has studied tutoring鈥檚 effectiveness.
鈥淧eople should be asking the question, 鈥楽hould we spend money on tutoring?鈥� That鈥檚 a totally fair question,鈥� he said. But findings that some pandemic-era tutoring programs aren鈥檛 effective right now doesn鈥檛 preclude them becoming more successful as districts refine and retool implementation, he said.
鈥淲e haven鈥檛 gotten there,鈥� said Kraft. 鈥淲e are tinkering toward that possible equilibrium. I think it would be a real loss in incremental innovation and improvement if we abandon that possibility.鈥�
From another vantage point, though, the problem is the same one that has continually plagued other major efforts to boost achievement, from class-size reduction to comprehensive school reform models to curriculum reform: scaling.
In practice, it鈥檚 often hard for schools to replicate the precise conditions that research on tutoring suggests can move the needle. Those include students learning from tutors one-on-one or in very small groups at least three times a week for about 50 hours of instruction over the course of a semester.
In a 2024 meta-analysis, Kraft and researchers Beth Schueler and Grace Falken examined 265 . On average, these studies showed that tutoring had a large, positive effect on student achievement.
But when they reran the numbers looking only at the studies that tested large-scale tutoring programs in the United States, and measured students鈥� outcomes via standardized tests, the effect size shrank by about a half to two-thirds, with larger tutoring programs having smaller effect sizes.
In part, that鈥檚 because larger programs tend to have higher student-tutor ratios and less time spent in tutoring than smaller programs, Kraft said. It鈥檚 also possible, he said, that 鈥渢he quality of delivery and implementation starts to slip when programs are scaling.鈥�
It鈥檚 easier to find 10 strong tutors, in other words, than 100.
鈥淲hat we鈥檙e learning from a really unbelievable wealth of rigorous studies, including the new study by SCALE, is a core insight in education,鈥� Kraft said. 鈥淚mplementation always trumps the base design of an educational intervention. You have to do something well for it to work.鈥�
What went wrong in one kindergarten tutoring program
The SCALE study examined an early literacy tutoring program with 270 kindergartners and 1st graders, and an early numeracy program that served about 1,000 kindergartners. Paraprofessionals served as tutors in both programs.
Neither group of students received as much tutoring as the programs suggested. Students in the reading program, on average, went through about 30% of the recommended sessions, while students in the math program went through about 36% of their lessons.
In part, this was due to school testing schedules, said Huffaker. Tutoring couldn鈥檛 start until students were evaluated with initial assessments, which delayed the beginning of the program to November. And within the math program, almost half of the students who were supposed to receive tutoring never got it鈥攁dministrators didn鈥檛 tell classroom teachers that those students were supposed to participate in the study.
But because the tutoring program was district-wide, the researchers were able to take advantage of differences in implementation in different schools. They found that when students received at least half of the sessions they were supposed to in the math program, achievement started to grow, and the effect got larger the more of the program that students attended.
The result delivers a strikingly simple insight, but one that could explain why some large-scale programs are found ineffective: 鈥淭he core idea and promise of tutoring is showing up, but only if students are receiving the tutoring,鈥� Huffaker said.
Dosage, tailoring both key factors to effective programs
In reading, the researchers found different patterns. Even when students completed more of the tutoring sessions, they didn鈥檛 see gains, on average.
Instead, lower-achieving students saw some growth compared to a control group, while students who started at a higher baseline didn鈥檛.
The literacy program featured evidence-based instruction, but every student started at the same place, with very beginning skills in phonemic awareness鈥攖he ability to recognize and manipulate sounds. It鈥檚 likely, Huffaker said, that most students didn鈥檛 reach enough material to grow much beyond where they started.
The math program, by contrast, allowed for more individualized instruction. 鈥淭here was less of a requirement to do every lesson in order, and more flexibility to skip around,鈥� she said.
Together, the findings suggest the importance of tailoring tutoring to student needs, and making sure they have enough time with tutors to make progress, Huffaker said.
Through interviews with school leaders, the researchers learned that campuses where students progressed through more of the tutoring program and were more successful had prioritized running the program effectively.
鈥淭hey instituted dedicated time every single day for the paraprofessional and the teacher to coordinate about instruction, and to make sure this is actually happening,鈥� Huffaker said.
Leaders need to allot time and effort to this kind of collaboration, she added, especially if the programs depend on people who already wear a lot of hats in the school.
