A school reform plan implemented in Michigan in 2012 didn't actually improve schools.
That’s according to a new working paper published this week by the National Bureau of Economic Research.
The study looked at reforms implemented as part of the state's waiver from requirements of the federal No Child Left Behind Act.
The waiver released Michigan from meeting student proficiency mandates under NCLB, and put in place new standards for reforming underperforming schools.
In Michigan, the bottom five percent of schools were labeled "Priority schools." Those schools had to implement certain changes, such as new administrators, new teaching strategies and new goals to measure progress. Another group of schools were labeled "Focus schools." These schools had the largest achievement gaps between high-performing and low-performing students.
Brian Jacob of the University of Michigan, and his co-author, Steven Hemelt at the University of North Carolina, compared results of these schools after the reforms were implemented with the results of schools in the state that narrowly missed being labeled either a Priority or a Focus school.
"So what we were able to do is just compare the schools on either side of that boundary," said Jacob.
For example, with Priority schools, the state automatically marked all schools in the bottom five percent of its top-to-bottom statewide ranking as Priority status.
"The fifth percentile schools had to undergo all of these additional reforms," said Jacob. "The sixth percentile schools didn't have to, but in many ways they were almost identical to the fifth percentile schools, so they make a really good comparison group."
Jacob and Hemelt used the same logic to look at Focus schools. Though the criteria were a little different for this group, they still looked at just the schools on either side of the cut-off for being labeled a Focus school.
Then they looked at what changed for both groups after the reforms went into place. Did the schools subject to mandatory reform efforts perform better than the schools that just barely avoided a Priority or Focus label?
No. According to Jacob and Hemelt, the reforms didn't make a difference.
Their study analyzed a number of possible changes, including student achievement on standardized tests, teacher staffing levels and student enrollment.
Almost none of the metrics were different in the schools that went through the reforms versus those that didn't.
Jacob said they did find some small change in Focus schools on math scores. It was short-lived, but the gap between high performing math students and low performing math students did appear to shrink—just not for the right reasons.
"Ironically, this reduction was caused by lower achievement among the top students," Jacob said, "as opposed to higher achievement among the bottom students."
It's not entirely clear why the reforms didn't work. But Jacob has some ideas.
He cited a report from the U.S. Department of Education that showed Michigan struggled to monitor schools that were subject to the reforms. And he says his own research shows that, when given a choice, many struggling schools chose the least dramatic turnaround options. For example, some schools may have replaced a principal at a struggling school, but not many teachers.
Taken together, Jacob said the problem may be that Michigan had weak implementation of its reform efforts. If true, the lesson he says isn't to institute a whole new set of reforms. It's just to stick with a plan and see it through.
"I think we just need more stability in the political and policy environment to allow the people on the ground to focus on the actual work at hand," he says.
Copyright 2017 Michigan Radio
