2022 Spotlight On Series:

Student Learning During the COVID-19 Pandemic

STUDENT ACHIEVEMENT BEFORE AND AFTER THE PANDEMIC

In March 2020, education in North Carolina changed dramatically as schools across the state were temporarily closed and then moved to remote learning in response to the COVID-19 pandemic.

Summative assessments were eliminated in 2019-20, but 2020-21 assessment data, compared with the pre-COVID 2018-19 data, give us a glimpse into the impact of these significant learning disruptions. As expected, student proficiency rates fell for students in all grades and in almost every subject.

 

 

Declines in student achievement varied across student subgroups and were greatest amongst the most at-risk students, including those who experienced homelessness and economically disadvantaged students.

 

 

Meanwhile, high school students’ achievement appeared to be impacted somewhat less than their elementary and middle school peers. Scores on high school End-of-Course assessments and ACT scores declined, but not as much as in the lower grades.

Note: The ACT is a cumulative test that measures learning across several years. 

 

 

THE IMPACT OF LOST INSTRUCTIONAL TIME

In March 2022, NC DPI’s Office of Learning Recovery and Acceleration released the COVID-19 Impact Analysis of Lost Instructional Time report, which assesses the extent to which students’ learning trajectories were altered by factors related to teaching and learning during the pandemic.  

In the charts below, a negative effect size indicates that students made less progress than expected in a typical year of schooling. Student progress was negatively impacted at all grades and in all subjects, except English II, which is typically taken in 10th grade.

 

 

Following the initial Lost Instructional Time Report, NC DPI issued a report that quantifies the interventions necessary for students to fully recover learning from the pandemic. Using research-backed methods that allow for the conversion of test score differences to units of school time, DPI estimated the number of months of intervention it will require students to recover learning lost during the pandemic. The additional learning time necessary ranged from 2.25 to 15.25 months and was higher in math than in reading.

 

 

THE IMPACT OF LOST INSTRUCTIONAL TIME BY STUDENT SUBGROUP

While the learning of all groups of students was diminished by the pandemic, some groups saw greater impacts than others, including American Indian students, Black students, and economically disadvantaged students.

 

 

Students attending high-poverty schools were also disproportionately impacted by the pandemic. 

 

 

THE IMPACT OF REMOTE LEARNING

Students who returned to the classroom for face-to-face learning earlier performed better than students who experienced higher numbers of remote learning days. The negative impacts of remote instruction were broad and pervasive, but most severely affected learning in grades 3 and 4 reading and in middle grades math. 

 

 

In August 2022, DPI issued a separate report examining the effect of home Internet access on student learning. DPI found that, in 2017-18, students with greater access to the Internet at home were more likely to meet or exceed growth expectations than students with less home Internet access. In 2020-21, the positive impact of home Internet access was greater in magnitude, signaling that reliable Internet access played a more prominent role in student success during the pandemic.

 

 

 

AND THERE’S MORE

To learn more about the impact of the pandemic on student learning, as well as DPI’s efforts to assess those impacts, check out these links: 

1. NC DPI’s Impact Analysis of Lost Instructional Time full report. 

2. A video that explains effect size. 

3. A primer summarizing the key findings of the Impact Analysis of Lost Instructional Time report. 

4. NC DPI Office of Learning and Recovery and Acceleration Research and Recovery Roundup blog. 

5. A white paper interpreting the results of the Impact Analysis of Lost Instructional Time report. 

6. A white paper quantifying necessary interventions for students to recover learning. 

7. A white paper highlighting the impact of home Internet access on student progress in English-Language Arts during the pandemic.

 

Sources: NC DPI Accountability Data Sets and Reports, NC DPI COVID-19 Impact Analysis of Lost Instructional Time, NC DPI White Paper: Recovery Interventions Require Effects Equivalent to Months of Additional School Time, NC DPI White Paper: Rural-urban differences in ELA progress and home internet access during pandemic