Are charter school networks really improving their students’ academic achievement?
That’s the central question to be addressed in what researchers involved believe is the first comprehensive study of the efficacy of charter-management organizations, or CMOs, which is scheduled to be launched later this year.
CMOs are nonprofit organizations that operate and replicate clusters of like-minded charter schools; among the better-known CMOs are the Los Angeles-based Green Dot Public Schools, the San Diego-based High Tech High, and the New Haven, Conn.-based Achievement First.
“There is a big black box between a student being affiliated with CMOs and that student’s learning,” said Paul T. Hill, the study’s principal investigator and the director of the Center on Reinventing Public Education, which is based at the University of Washington in Seattle. “We’re trying to fill in the box.”
The three-year longitudinal National Study of CMO Effectiveness, announced this month by the San Francisco-based NewSchools Venture Fund entrepreneurial philanthropy and the Seattle-based Bill & Melinda Gates Foundation, will cover almost 200 schools within 33 CMOs in 12 states.
“Although many studies have been done on the effectiveness of charter schools, this is the first expansive study on CMO effectiveness,” Vicki L. Phillips, the Gates Foundation’s education chief, said in an e-mail.
Starting next fall, investigators will research in earnest what’s projected to be a three-pronged look at the efficacy of as many nonprofit charter networks as will consent to be studied. The Gates Foundation expects that the study will cost more than $3 million.
The study will focus on the academic impact of CMOs, their best practices, and how CMOs affect and are affected by their regulatory, competitive, and other contexts, said Brian P. Gill, a senior social scientist at Mathematica Policy Research Inc., of Princeton, N.J., who will co-lead the study.
According to Mr. Gill, the study will use both experimental and what he called “state-of-the-art quasi-experimental” data.
For example, in studying the effectiveness of a CMO in a given area, the study will make use of comparative achievement data from students who gained admission to a given charter school vs. students who did not, in cases where an oversubscribed school employed an admissions lottery.
“The problem, though, is that not every charter school is oversubscribed, so we can’t do this everywhere—we’re going to do it opportunistically,” Mr. Gill said.
Ultimately, he said, achievement will be measured student by student, using respective scores on state standardized tests.
Criticism Seen
Mr. Hill acknowledged the challenges of comparing student and CMO achievement in different ways.
“We think we’re brave as hell,” he said. “Every one of these comparisons is imperfect in some way—you have to be very clear about that.” But he said that the comparisons will “be Granny Smiths-to-Galas, not apples to oranges.”
Marc Dean Millot, the editor of the Alexandria, Va.-based online newsletter School Improvement Industry, who previously worked with the Center on Reinventing Public Education and Mr. Gill, said he would construct the study differently. “I think you’d want to look at similarly situated traditional schools,” in addition to comparing CMO-affiliated charters with non- CMO charters, said Mr. Millot, who also maintains the independent blog Edbizbuzz on edweek.org.
He also questioned why foundation research on effectiveness was only being initiated now, given the philanthropic money that’s been poured into CMOs over the years. The Gates Foundation reports having invested more than $105 million in CMOs since 2000.
“This kind of research should have been going on from day one,” Mr. Millot said. “There is no excuse from a social-purpose standpoint that, ‘Here’s all this money, but we’re not going to open the stove until 10 years into the cooking process.’”
Joanne Weiss, a NewSchools Venture Fund partner and its chief operating officer, said her organization, which will oversee the study, has been gathering achievement, cost-effectiveness, and other data all along.
“But that’s really, really different from a research study,” she said, adding that to do a valid research study, the study’s sample size needs to be big enough.
