Culturally and Linguistically Responsive Assessment: Myths and Opportunities

April 21, 2022

By Molly Faulkner-Bond, Senior Research Associate with English Learners and Migrant Education at WestEd

Exciting things are happening in the world of assessment right now—an awakening of sorts. Just as many organizations and fields are beginning to wake up to the reality of systemic race-based inequity, the field of assessment is examining its practices and values through this lens.

As evident in recent publications, conference presentations, and webinars, assessment professionals are engaged in ongoing efforts to articulate and understand how education assessment might be perpetuating systemic inequalities—and, more importantly, how to improve assessment practices to better serve all students.

Despite this excitement about addressing culture and language in assessment, there is also trepidation. Some assessment professionals question the wisdom or value of shifting practices towards cultural and linguistic inclusivity. The field values standardization and comparability. Both of these principles seem threatened if assessment instruments are infused with the diverse languages and cultures of American students’ lived experiences.

It also may concern some that the field is still developing consensus on terms and definitions. What does “culturally and linguistically responsive assessment” really mean? In this post, I use the phrase to mean assessment practices and instruments that reflect students’ diverse experiences, interests, and values in authentic, positive ways.

While I empathize with some concerns that are commonly expressed about shifting assessment practices, I believe many of the concerns arise from myths and misconceptions about what is meant, who is affected, and what is at stake in maintaining versus transforming education assessment practices to be more inclusive.

In the remainder of this post, I describe and then counter four common myths that underlie some of the concerns I have heard expressed about culturally and linguistically responsive assessment practices.

Myth: Current assessments are culturally neutral or do not reflect any specific culture.

The whole point of a standardized assessment, some might argue, is that it reflects no one and everyone—it is generic, universal, neutral. This neutrality is central to our ideas of fairness: If the test reflects no one, then it is equally fair for everyone. The Standards for Educational and Psychological Assessment clearly state that “test developers are responsible for…minimizing the potential for tests’ being affected by construct-irrelevant characteristics, such as linguistic, communicative, cognitive, cultural, physical, or other characteristics” (p. 64).

Reality: Although this pursuit of neutrality makes sense, it’s unclear if assessment developers are actually achieving it in the way they create assessments. As scholars of race have argued (in other fields and in educational assessment), one of the key ways that “whiteness” functions in American culture is to occupy a position of alleged neutrality. Because the experiences of White Americans are almost universally centered in the country’s institutions and decision-making, the White perspective and White culture are treated as defaults—and other cultures and perspectives as diversions from that default. With this framing, the practice of stripping out content that registers as “cultural” from an assessment serves to essentially remove all racial and cultural perspectives and experiences except for the White default. In other words, the tests that people often think of as “neutral” are actually reflecting a single racial perspective: the White perspective.

Myth: Truly responsive assessments would need to be differentiated and tailored to each group (e.g., “the Black test,” “the Asian test”) or would need to reflect multiple cultures on each test, which would divide students or make everyone feel alienated at some point.

Reframing the first myth means recasting “neutral” assessments as ones that are actually culturally “White” and homogeneous. This second myth assumes that the solution to White homogeneity is other kinds of homogeneity—and that maintaining homogeneity is about protecting students from discomfort or harm that might arise from seeing unfamiliar words, phrases, or content.

Reality: Is it jarring to see a word, phrase, or name on an assessment that you’re not personally familiar with? Sure—and for that reason, any conversation about culturally and linguistically responsive assessment should include bigger picture thinking about classroom learning, formative assessment practices, and other inputs that can prepare students for what they might see on a test. But educators also already have and deploy techniques to help make the unfamiliar familiar to students who may need it (e.g., by providing word-level translation on a non-English word used in a story, or by providing glossaries and translations for nonessential vocabulary for students who are English learners).

In considering the potential harms of seeing unfamiliar content on a test, assessment developers and educators should also remember to weigh the potential benefits and value of learning about other cultures. If a student never sees content outside of the student’s own culture, a test question about another culture might indeed be challenging. But should schools limit the diversity of languages and cultures in classrooms to prepare students for a homogeneous, “neutral” test? Or should students’ learning experiences be diversified with a range of perspectives and experiences—and then tests built to reflect that diversity?

Myth: Diversifying language and culture on a test will add unnecessary length and/or complexity and may thus differentially affect vulnerable groups such as students who are English learners.

My work focuses heavily on students classified as English learners, so a concern I often hear raised is that culturally and linguistically responsive assessments may be more difficult or too difficult for these students. The idea is that the additional context and language that are needed to present non-White cultures authentically will introduce additional length and complexity to items, and such additions will be particularly challenging for students who are still developing their English skills. Essentially, those voicing this concern may fear that assessments designed to be more inclusive could unintentionally have the opposite effect for some of the most vulnerable students.

Reality: Diversifying language and culture on a test can include acts as simple as diversifying the subjects and topics for pictures, passages, stems, and other stimuli in an assessment. For example, a reading passage about a White male scientist may be replaced with a reading passage about a Latina astronaut. Or a math problem about units of measurements might focus on making banana pudding instead of apple pie. None of these changes should affect an item’s length or complexity.

However, sometimes more information or language is needed for conveying sufficient context about any culture or experience. In such instances, it is worth asking: If students may struggle to access rich content and complexity, is the best approach to strip out that richness and complexity, or to find better ways to scaffold students’ access to the content? My colleagues at Quality Teaching for English Learners (QTEL) at WestEd will affirm that the answer is unquestionably the latter—and they spend great time and effort teaching educators and leaders across the country how to do this scaffolding.

Myth: Making large-scale standardized assessments culturally and linguistically responsive is not possible or advisable.

Some individuals may be on board with creating more culturally and linguistically responsive assessments but argue that they are more appropriate for classroom-based assessment. It is true that the classroom is a more conducive environment to creating and implementing assessments that allow students the time and supports to really engage with rich content that may spark conversations, disagreements, and growth opportunities. And infusing that same richness into large-scale standardized assessments may be somewhat harder—particularly if the testing time is limited for these kinds of instruments.

Reality: While I agree that there are challenges in applying some of these principles to large-scale standardized assessments, that is not a reason to give this category a pass. Rather, assessment developers should think of these practices on a continuum and acknowledge that different assessments afford different opportunities. Culturally and linguistically responsive assessments will look different at different levels of a comprehensive system. There is more flexibility at the classroom level, where students can co-create assessment opportunities, topics, and success criteria with teachers (e.g., via formative assessment practices, portfolio-based assessments, and assessment systems), but there are steps that can and should be taken at every level of an assessment system to ensure that assessments are not erasing or excluding most groups.

Next Steps and Opportunities

How can educators, researchers, assessment professionals, and technical assistance providers help make assessment more inclusive, more diverse, and more responsive to students’ lived experiences? I don’t presume to have the answers to these questions, and indeed—I think articulating this work is an important collective activity for the assessment field in the coming years. We all need to work together to think about what we need to know about developing more equitable assessments, how to collect that information, and how to apply those findings to our own practices.

Toward that goal, I offer two initial steps that assessment developers can take right now that can help them start to make progress towards this larger goal.

First, review your own assessment processes and products, including who is involved in creating and reviewing assessments, and at what point in the process. The purpose of this activity is to start to quantify the diversity (or lack thereof) of your existing items and assessments, as well as who is involved in creating and reviewing those materials. Questions to ask in this process might include:

• How many items can you identify on your instruments or in your item bank that explicitly center the experience of an individual who is neither White nor “neutral”?
• What is the racial and cultural composition of your item development team, including artists and passage writers?
• Do individuals who represent or resemble the end users of your items—i.e., students, teachers, communities, families—ever see them before they encounter them on a test form?
• Is there any mechanism for these users to share feedback or reactions with you?

Second, using the information you gather from the first step, identify points in that process where you can increase the diversity and amplify the input of underrepresented groups by inviting them into the process to provide input and feedback. The less diverse the development teams, or the later in the development process that diverse voices are invited to the table, the more likely you are to miss out on crucial opportunities to introduce new perspectives, content, and approaches into assessment instruments.

Efforts to create culturally and linguistically responsive assessments also afford the assessment community some great opportunities to achieve goals everyone presumably shares, including:

• Diversifying the assessment field by inviting in students, parents, and communities that are historically underrepresented in this field. These individuals may create content, review content, or—even better—enroll in graduate programs and apprentice with test development teams to become professionals themselves.
• Improving the validity of assessment products and practices, particularly for populations that have tended to underperform despite having comparable underlying distributions of skills and abilities.
• Supporting the improved achievement of all students, and particularly those from historically underserved backgrounds.

Change is never easy or straightforward. But increasingly the assessment field is ready to change its practices to be more inclusive. WestEd is proud to be a part of that work.

---

Molly Faulkner-Bond specializes in improving policies, assessments, and programs for English Learners. Through research and technical assistance, she works to increase the use and utility of education research and enhance education outcomes for linguistically and culturally diverse learners.