Assessment and Accountability Systems in a Post-COVID World

April 1, 2022

By Andrew Latham, Senior Managing Director at WestEd

Although Omicron cases are finally beginning to recede, the United States still has far to go before declaring victory over COVID. The pandemic has had a major impact on state assessment and accountability systems, bringing into question how much faith educators can place in assessment data gathered from non-representative samples of students who have had widely varying opportunities to learn.

WestEd’s assessment teams recently convened to discuss the lessons we have learned during the pandemic and their implications for assessment and accountability systems in the future.

Challenges for Summative Testing

The pandemic has presented extraordinary challenges for both teachers and students, particularly students from underserved groups who have had unequal access to classroom learning opportunities and internet connections for remote instruction.

Statewide assessment data from this period are of limited utility because, in many states, far fewer students participated in annual summative testing than the traditional minimum of 95%, bringing into question how representative scores are of the full student population.

States can, however, glean from the summative tests that were given, as well as from local district assessments, that nearly all student learning in the core disciplines of math and English language arts suffered.

States likely won’t know the full extent and nuances of this reduced learning until they can once again give annual summative assessments to at least 95% of all students, but they can start thinking now about what the first generation of post-pandemic assessments should look like.

Using District-Level Assessments

Most states have not assessed all their students as they would in a typical year. Without solid baseline data, measurement of important outcomes, such as average student learning gains, is difficult. This challenge is exacerbated by issues around reduced opportunities to learn; if summative assessments are designed to measure students’ learning against a set of grade-level standards, but the pandemic restricted learning opportunities for at least some students against some subsets of standards, the summative assessments aren’t an accurate measure of what students can be expected to know and do after a typical year of instruction.

One way around this issue is to look at results from assessments given at the local level during the pandemic and see how they compare to performance on those assessments from a normal year. These types of comparisons can present their own measurement challenges—for example, within a state, local assessments can vary widely in terms of their difficulty, standards and content covered, and score report scales. WestEd has worked with several states in the past year to identify valid and meaningful trends contained in the data from disparate assessments given by districts across those states.

Four Ways to Enhance the Validity and Utility of Assessment Results

Although these assessment and measurement challenges are numerous, they are not insurmountable. In working with our state and district clients, WestEd’s assessment teams emphasize four ways that states and districts can enhance the validity and utility of their assessment results:

1. Seek coherence in non-summative assessments. While locally used assessments from commercial providers can vary in their design, scaling, and reporting, they can provide a partial picture of state-level performance. For example, districts that have used the same local assessments for years can compare their students’ pre-pandemic performance on those assessments with data from 2020 and 2021 to gauge roughly how much the pandemic has impacted learning.
2. Consider alternative models such as through-year assessment. Through-year assessment models have long been seen as an appealing alternative to traditional summative assessments, since they offer the possibility of assessing students closer to when their learning takes place, as well as the promise of receiving results in time to influence subsequent instruction. These models have proven more difficult to implement in practice than in theory, however; for example, the original PARCC model contained through-year elements that were later dropped. Today states such as Nebraska and Georgia are actively piloting and considering through-year models, while other states, such as Florida and Illinois, have signaled their strong interest in pursuing such models in the future. When done thoughtfully, such models offer the promise of assessment in service of instruction, and less emphasis on a single year-end test that significant portions of students may opt out of or miss for other reasons, including pandemic-related reasons. But these models also require years of planning and piloting before operational adoption, so pursuing this type of solution would require a major commitment from the state.
3. Provide assessment literacy training for teachers. If teachers do not understand or value the assessment information they are receiving about their students, they will have wasted considerable instructional time gathering information with limited instructional utility. Now, more than ever, it is essential that teachers are provided with guidance on exactly what students’ assessment scores mean and how those scores can be used to analyze and design relevant instruction. At WestEd, we find the most effective assessment literacy training focuses on helping teachers understand what is being assessed, the expectations for student performance (i.e., what does it mean for a student to be proficient?), and the instructional resources and strategies that they can use to address student learning gaps.
4. Analyze the underlying factors driving student performance. Summative assessments can shine a light on how students are performing, but not necessarily on why they are performing that way. For example, while they can show that student performance has declined during the pandemic, they don’t reveal what factors have had the most impact. Was increased absenteeism to blame? What about a lack of access to online instruction? Or insufficient access to a teacher or parent for assistance during remote instruction? States can use summative assessment results to identify areas for additional investigation; subsequent analysis of how all the various pandemic-related instructional changes interact to affect student performance is essential to understanding how best to help students recover from lost learning opportunities.

WestEd remains committed to and supportive of fair, valid, and thoughtful summative assessment and accountability programs. Although all data collected during the pandemic must be evaluated with caution and within students’ learning contexts, summative data still provide the clearest picture of how the students in any given state are performing as a whole and in subgroups.

This information is essential for helping states to establish high but attainable learning objectives for their students. However, summative assessments are only one tool for teachers and schools. Educators must identify the specific factors that cause disparate and declining student results, then work with teachers to determine how best to address them.

How WestEd Can Help

WestEd helps clients develop comprehensive standards and standards-based assessment and accountability systems for students of all ages and learning paths.

We collaborate with clients nationwide, providing policy guidance and assessment design services responsive to local contexts. We create customized, research-based solutions to gauge student performance and promote engaging classroom settings in early childhood, K–12, postsecondary, and career technical education.

Learn more.

Additional Resources 

Publications and Webinars

Connect With Us

How is your school, district, or state working to address assessment and measurement issues related to the pandemic? Join the conversation on Twitter, Facebook, and LinkedIn.

Andrew Latham, a national leader in educational assessment for more than two decades, is the Senior Managing Director responsible for WestEd’s assessment design, development, research, and innovation portfolios. Latham works closely with national organizations, states, schools, and districts to develop innovative solutions for assessing and setting standards for students and teachers.

The Center for Standards, Assessment, and Accountability (CSAA) at WestEd strives to support balanced, coherent, and efficient systems of teaching and learning. For this reason, CSAA provides a comprehensive portfolio of research-informed resources, high-quality technical assistance, and services designed to support meaningful improvement of student learning. Learn about Standards, Assessment, & Accountability at WestEd.