Includes updates and/or revisions.
A federal advisory panel said yesterday it will urge Congress to “refrain from eliminating funding for Reading First,” as House and Senate appropriations subcommittees have recommended, because the panel’s members believe a recent evaluation has caused confusion and misperceptions about the program’s effectiveness.
“We have concerns about the limitations of the study that preclude drawing firm conclusions,” Susan Brady, a member of the Reading First Advisory Committee, said at the meeting. The statement will be forwarded to members of the Senate Appropriations Committee, which is expected this week to make its recommendation for funding the program in the fiscal 2009 federal budget.
It outlines three concerns: “limitations surrounding the design, methodology, and analysis of the Reading First Impact Study; unwarranted inferences drawn from the interim report [of that study]; and lack of consideration of other sources of information about the effectiveness of Reading First.”
“Certainly, any decisions about funding,” the statement continues, “should consider multiple sources of data.”
The House subcommittee’s version of the budget would kill the program, based in part on the interim report, which found that the federal funding has had no significant impact on students’ reading comprehension. The Senate subcommittee that handles education appropriations also voted to end funding for Reading First, but that panel didn’t cite the interim report. (“Senate Panel Also Votes to Kill Funds for ‘Reading First’,” June 24, 2008.)
The independent advisory panel of reading researchers, which was appointed by U.S. Secretary of Education Margaret Spellings last year, is charged with reviewing ongoing issues related to the program and making recommendations to the Department of Education.
After a briefing on the Reading First Impact Study Interim Report here, the members lamented that the findings have been broadly interpreted to mean that, after a $6 billion investment, the program has failed to raise reading achievement. The preliminary results were based on a single measure of reading comprehension, and not the basic, early skills that are the main targets of instruction in the early grades. (“‘Reading First’ Research Offers No Definitive Answers,” June 4, 2008.)
“I don’t think we should misinform the public and lead them to believe that this was a solid study, because for whatever reason, it wasn’t,” said panel member Frank Vellutino, a prominent reading researcher at the State University of New York at Albany. “I’m not trying to spin the data, ... but to allow people to spin their own tales, including members of Congress who want to scuttle the program,” is not acceptable, he said.
Past Criticisms Cited
Rep. David R. Obey, D-Wis., the chairman of the House Appropriations Committee, has pushed to slash the program over the past year, citing a series of reports in 2006 and 2007 by the Education Department’s inspector general. Those reports found that Reading First was mismanaged and that federal officials and consultants involved in the program appeared to have used their influence to promote programs and assessments that they favored.
Reading First “has been plagued with mismanagement, conflicts of interest, and cronyism, as documented by the inspector general,” Rep. Obey said during the subcommittee meeting. “Moreover, a scientifically rigorous study released by the Department of Education found that the program has no discernible impact on student reading performance.” (“House Panel Would Kill ‘Reading First’ Funding,” June 19, 2008.)
The long-awaited Reading First Impact Study Interim Report, released May 1, caused a backlash among supporters of the program, who point to district and state test results as evidence that it is having an effect. Much of that data is not comparable across states and has not been subjected to rigorous analysis.
Positive Changes?
Several independent surveys have found that most participating schools have noticed positive changes in instruction and students’ test scores. An analysis of state test-score data, conducted by the American Institutes for Research and released by the department last week, shows that a majority of states have seen gains in reading fluency and comprehension in Reading First schools.
The gains were across grades 1, 2, and 3, and most subgroups of students. The analysis is not a rigorous study, however, and averages data that is generally not comparable because all states do not use the same tests and they may set different benchmarks for defining proficiency. The data are from annual performance reports submitted by each state and have not been verified as accurate.
The unusual design of the $31 million interim impact study—a regression, discontinuity analysis—has drawn criticism for comparing Reading First schools across 18 sites with nonparticipating schools in the same districts. Some experts have argued that the sample is “contaminated,” because the non-Reading First sites have adopted many of the principles and practices of those that are in the program, but have not benefited from annual federal grants.
‘Hit the Pause Button’
Many of the panel members suggested that Congress needs to have additional information, including data that paint a more positive picture of how the program is going. But they also stated that they wanted their input to be neutral, and to maintain the panel’s advisory role without crossing over into advocacy. Several members pressed for immediate action, however.
“Can we march over to the Senate” and present the committees with additional information, asked Ms. Brady, a panel member and a senior scientist at the Haskins Laboratory in New Haven, Conn.
“Unless Congress gets different information, this program is going to go by the wayside,” said Laurie E. Cutting, the director of the education- and brain-research program at the Kennedy Krieger Institute in Baltimore. She noted the the committee had to prepare its statement quickly so that it might have an influence on the budget discussions. “Can’t we say, ‘Hit the pause button please; it’s not really appropriate to be making these decisions yet.’ ”
Several members said they were frustrated that the interim study was having such an effect on decisionmakers, while a trove of state data has yet to be mined. Moreover, they said, a draft of the final impact report, which includes additional data on student achievement and teacher practice, is complete and going through peer review, yet not set for release until the fall.
“The timing of this is so off,” said Nonie K. Lesaux, a professor of human development and urban education advancement at Harvard University. “It’s crazy to think that the final report is drafted and sitting somewhere while its fate is being debated.”
All the members agreed it was appropriate to alert federal lawmakers that they think it is premature to make significant decisions about the program until more evidence is weighed.
The panel will form subcommittees to draft an in-depth analysis of the study’s findings, and to clarify for Congress and the public what it does and doesn’t say about the program’s effectiveness.
