The big-name companies in education have a lot riding on the credibility of their products, services, and technologies. And that credibility comes with a price. Serious efficacy studies can start as high as $150,000, a price tag that keeps most smaller market players from commissioning similar studies.
“We all have a responsibility to ensure that our products and services obviously fulfill the educational promise we’re making,” said Francie Alexander, senior vice president of Scholastic Education and chief academic officer for global book publisher Scholastic, Inc., based in New York City. To that end, Scholastic has a five-stage research life cycle that starts with foundational and formative research and ends, depending on the program or product, with ongoing replication studies.
Continual data analysis is critical for maintaining that educational promise, said Ms. Alexander, referring to a necessary addition in 2008 to the company’s READ 180 reading-intervention software program, which launched in 1999 and is designed for students in grades 4 and higher. Of 23 major studies of READ 180, only two indicated no positive effect; further digging found problems with implementation and fidelity.
Still, through its work with school districts, Scholastic learned that there were large numbers of students who read below the 2nd grade level and needed foundational help.
In response, the company developed a prequel, System 44. A study conducted by RMC Research, based in Portsmouth, N.H., during the 2011-12 school year showed that System 44 helped students with learning disabilities catch up to their peers in reading and outperform peers in a control group who received other interventions.
READ 180 has earned validation from the U.S. Department of Education’s What Works Clearinghouse, whose reports are known as education’s highest standard in program evaluation.
The textbook-publishing giant Pearson, meanwhile, has a design-and-development research life cycle that includes some 30 types of in-house and independently conducted studies, from literature reviews to randomized control trials. Each academic year, the British company, with U.S. headquarters in New York City, hires third-party evaluators for eight to 10 randomized control trials on materials that focus on critical areas of instruction and are designed around national programs.
In January, the clearinghouse reported that an independent study of Pearson’s enVisionMATH elementary school curriculum—which concluded that the curriculum increases student achievement above and beyond other K-6 math programs—met its evidence standards “without reservations.” The independent study was done by Planning, Research & Evaluation Services Associates, based in Jackson, Wyo.
The clearinghouse gave another of Pearson’s elementary school math programs a thumbs-up in March.
‘It’s About Longevity’
Steven Ross, the evaluation director for the Center for Research and Reform in Education at Johns Hopkins University in Baltimore, explained the importance of research about company products, especially from a district’s perspective.
Read a related story, “Schools Vet Ed. Company Research With a Critical Eye.”
“If I’m interested in Success for All, and I find five randomized control trials showing that this particular approach to reading produced significant effects, I’m pretty happy,” he said. “I know it’s not a fly-by-night program. That’s the value.”
Cisco Systems Inc., based in San Jose, Calif., sponsors broad-view research driven by market trends. For example, the networking-equipment manufacturer, which provides video technology and services, last year commissioned the independent analyst firm Wainhouse Research to study the impact of video streaming and broadcasting in education.
Wainhouse, located in Duxbury, Mass., concluded that the use of video in education is a fledgling practice that can have a powerful impact on student engagement and performance.
“We tell our customers, ‘This is what’s really important out there ... and this is how we can address that for you,' " said Renee Patton, the U.S. public-sector director of education for Cisco, which also conducts in-house research.
What does a company do if research results aren’t flattering? When there’s a major flaw in the product or with the study’s execution, results typically are considered part of a formative study, one that will help the company make modifications for better results, according to Dallas-based Rob Foshay, who formerly managed the portfolio of independent research for Texas Instruments and Plato Learning, now known as Edmentum.
Not only can larger companies afford rigorous research to back up their claims, they also have recognizable brands—and that prominence alone makes it easier for some districts to plunk down money on a large purchase.
“To me, it’s about longevity,” said Bobby Blount, the assistant superintendent for accountability, technology, and strategic planning for the 39,000-student Cherokee County, Ga., school district. “Name recognition instills a little confidence that the company has been around a long time, and you know they will be [around even longer].”
