How Education Technology Can Make the Grade

April 15, 2017

Classrooms throughout the United States face an explosion of new digital technologies. Education technology is a burgeoning market — expected to grow to $252 billion globally by 2020.¹

“In fact, teachers and administrators are often overwhelmed by the number of ed tech products available in the market,” says Michelle Tiu, Senior Research Associate in the Science, Technology, Engineering, and Mathematics (STEM) program at WestEd. Whether in the form of web-based instruction, mobile apps, or virtual and augmented reality, technology tools have the potential to improve teaching and learning and to help make education equitable for all students, says Tiu, “but how do you figure out the signal from all that noise? How do you know which products can be effective?”

At WestEd, Tiu coleads a portfolio of projects that have evaluated more than 50 education technology products in recent years, generating insights into what makes for the most effective tools — the ones that can really help students succeed.

Needed: Deep Understanding of Education and Technology

Over the years, Tiu and colleagues have found that the highest-quality ed tech tools incorporate four specific areas of expertise. “The most effective ed tech tools are built by developers who have a deep understanding of learning science, have a strong command of subject matter content, take advantage of the affordances that technology can provide, and understand the day-to-day classroom experience,” says Tiu. When evaluating ed tech products, WestEd’s staff use these four kinds of expertise as lenses to focus on quality.

“The most effective ed tech tools are built by developers who have a deep understanding of learning science, have a strong command of subject matter content, take advantage of the affordances that technology can provide, and understand the day-to-day classroom experience.”

Attention to learning science (an interdisciplinary field concerned with improving instructional methodologies) and deep content expertise can be seen in an ed tech product’s ability to individualize a user’s experience and to turn the student’s mistakes into learning opportunities. “Ed tech products that are built on a strong foundation of learning science and content tend to anticipate where students commonly have misconceptions, provide scaffolding to help them learn, and offer effective instructional opportunities that are grounded in research,” says Tiu.

Supporting learning in this way is a big advance over older computer tools, says Steve Schneider, Senior Program Director of WestEd’s STEM program. He explains how ed tech products are taking advantage of the affordances of technology: “In the past, when you got an answer wrong, the software usually just gave you another question, and without any hints.” Newer, more effective programs are more adaptive, adjusting the difficulty of problems to continually challenge each student — no matter their skill level.

Schneider notes that even when older software provided individualized assistance to students who had fallen behind, the approach was one of repetition, similar to “drill-and-kill” multiplication tables. “Now, personalized tools enable students to work at their own pace within a piece of curriculum,” he says. Students might read the same nonfiction article, for example, but at different reading grade levels. “Because the software automatically assesses and adjusts to various levels of reading proficiency, students can advance at their own pace, which helps free up the teacher to possibly spend more time working with students who need more attention,” says Schneider.

In addition, ed tech products can help teachers become more efficient in the classroom, says Tiu. Technology might assist teachers with time-consuming and repetitive tasks, freeing their time to potentially focus on providing more one-on-one, targeted instruction with students. For example, technology can assist with formative assessment. Instead of teachers’ having to manually grade and analyze data by hand, many ed tech tools that embed frequent checks for understanding can easily handle collecting, analyzing, and summarizing data, providing real-time reports that allow teachers to quickly understand an individual student’s areas of strength and areas for improvement.

Adding Value with Research

Not all companies, particularly small ones, have the full range of expertise in-house to put together all of the pieces of learning science, deep content knowledge, classroom expertise, and technological know-how. Accordingly, companies may bring in outside expertise, particularly in research and development, through programs such as the U.S. Department of Education’s Small Business Innovation Research (SBIR) program.

Kiko Labs, a company that created a game-based app to develop young children’s executive function skills, has drawn on SBIR funding to partner with WestEd for research and evaluation support in the company’s product development process. “We recognized that our strength was with product design and that we didn’t have the capacity to add extensive internal research — especially given the small size of our company,” says Grace Wardhana, CEO of Kiko Labs.

Kiko Labs’ Thinking Time product arose out of a recognition that executive functioning is critical to children’s development and school readiness.² The best window for developing executive functioning skills is from age 3 to 6, but “research shows that these skills are not guaranteed — they don’t necessarily materialize with maturity,” says Wardhana. “Children with challenges don’t usually get the support they need to address these deficits, and the research shows that children from lower-income communities are most at risk. We wanted to create a solution to assess children and help them practice these skills early.”

The resulting suite of games targets the development of executive functioning and reasoning in pre-K students. In early phases of development, WestEd was extensively involved in conducting formative research to ensure that the games could be effective in helping children learn these skills.

As products become more fully developed, they are ready for larger-scale feasibility testing in classrooms, where researchers gather data about whether teachers can integrate the tool into classroom routines and practices, or whether the tool is too difficult to implement and will tend not to be used. Tiu notes that it is important to ask, “Can the intended audience use it — whether student, teacher, or administrator — and does it achieve the desired outcomes?” If schools focus more on questions of efficacy, she adds, the better products will rise to the top. Another question to consider is whether use of the tool can be scaled up. “The ability to use a product over multiple districts is a potential asset of these kinds of technology,” says Wardhana.

WestEd is continuing its research on Thinking Time through a randomized controlled trial intended to test the product’s efficacy in preschool classrooms and to measure its potential impact on children’s cognitive skills.

Extending Capacity Through New Partnerships

Building on the research and evaluation assistance that WestEd has provided to product developers through the SBIR program, WestEd began a partnership with NewSchools Venture Fund. A nonprofit that has invested in education entrepreneurship since 1998, NewSchools seeks to increase equity and improve outcomes for low-income students and students of color. “As a thought partner to NewSchools, we offer expertise across the field of education, help them understand the ed tech research landscape, and connect NewSchools grantees with experts across education arenas,” says Tiu.

A program called NewSchools Ignite offers grants of $50,000 to $150,000 as well as other support to technology developers, says Cameron White, Associate Partner for NewSchools Venture Fund.

“Talking with educators, researchers, entrepreneurs, and other experts, we identify market gaps where there is potential for technology to significantly improve educational outcomes,” says White. Every six months, NewSchools synthesizes its findings into an entrepreneurial call to action, or “challenge,” and selects up to 15 companies and nonprofits developing technology that can address the needs identified through market research — a total of 10 challenges over the course of five years.

WestEd has played an integral role throughout the first three NewSchools Ignite challenges, which have focused on science, middle and high school mathematics, and English language learning. WestEd has served as a research partner for each of the challenge winners.

In partnership with NewSchools, a WestEd researcher and an educator provide the challenge winners with evidence-based formative feedback on their products, which forms the foundation for small-scale studies. Tiu explains that these types of studies help NewSchools challenge winners to iterate and improve upon their products, using feedback collected directly from teachers and students in the classroom.

WestEd offers a menu of small-scale study designs, including focus groups, subject-matter reviews by experts, and usability and feasibility testing, according to Schneider. “For more advanced products, we look at student gains in learning through implementation studies or studies of promise,” he says.

Both WestEd and NewSchools are looking to expand their support and further spread what they’ve learned from working with ed tech developers. The two organizations have been awarded a grant from the Small Business Administration’s Office of Investment and Innovation Growth Accelerator Fund. It will help build an online portal, launching in 2017, that will more broadly share key learnings with entrepreneurs, funders, and educators.

Notes:
¹ MarketWatch: “Global Report Predicts EdTech Spend to Reach $252bn by 2020.”
² Kiko Labs website: “Science.”

The U.S. Department of Education’s Institute of Education Sciences (IES), NewSchools Venture Fund, and Small Business Innovation Research (SBIR) grants from IES, the National Science Foundation, and the National Institutes of Health have funded WestEd’s work in evaluating education technology tools. The contents of this article do not necessarily reflect the views or policies of these funding agencies.