Schools That Transform

David Brooks reports on some surprisingly large impacts from a randomised trial of New York “Harlem Children’s Zone” schools. The effects are so big that I find it hard to imagine they’re readily replicable, but it’s tantalizing stuff.

The fight against poverty produces great programs but disappointing results. You go visit an inner-city school, job-training program or community youth center and you meet incredible people doing wonderful things. Then you look at the results from the serious evaluations and you find that these inspiring places are only producing incremental gains.

That’s why I was startled when I received an e-mail message from Roland Fryer, a meticulous Harvard economist. It included this sentence: “The attached study has changed my life as a scientist.”

Fryer and his colleague Will Dobbie have just finished a rigorous assessment of the charter schools operated by the Harlem Children’s Zone. They compared students in these schools to students in New York City as a whole and to comparable students who entered the lottery to get into the Harlem Children’s Zone schools, but weren’t selected.

They found that the Harlem Children’s Zone schools produced “enormous” gains. The typical student entered the charter middle school, Promise Academy, in sixth grade and scored in the 39th percentile among New York City students in math. By the eighth grade, the typical student in the school was in the 74th percentile. The typical student entered the school scoring in the 39th percentile in English Language Arts (verbal ability). By eighth grade, the typical student was in the 53rd percentile.

Forgive some academic jargon, but the most common education reform ideas — reducing class size, raising teacher pay, enrolling kids in Head Start — produce gains of about 0.1 or 0.2 or 0.3 standard deviations. If you study policy, those are the sorts of improvements you live with every day. Promise Academy produced gains of 1.3 and 1.4 standard deviations. That’s off the charts. In math, Promise Academy eliminated the achievement gap between its black students and the city average for white students.

Let me repeat that. It eliminated the black-white achievement gap. “The results changed my life as a researcher because I am no longer interested in marginal changes,” Fryer wrote in a subsequent e-mail. What Geoffrey Canada, Harlem Children’s Zone’s founder and president, has done is “the equivalent of curing cancer for these kids. …

These results are powerful evidence in a long-running debate. Some experts, mostly surrounding the education establishment, argue that schools alone can’t produce big changes. The problems are in society, and you have to work on broader issues like economic inequality. Reformers, on the other hand, have argued that school-based approaches can produce big results. The Harlem Children’s Zone results suggest the reformers are right. The Promise Academy does provide health and psychological services, but it helps kids who aren’t even involved in the other programs the organization offers.

… Basically, the no excuses schools pay meticulous attention to behavior and attitudes. They teach students how to look at the person who is talking, how to shake hands. These schools are academically rigorous and college-focused. Promise Academy students who are performing below grade level spent twice as much time in school as other students in New York City. Students who are performing at grade level spend 50 percent more time in school.

For more wonkish readers, the abstract of the paper is over the fold.

Are High-Quality Schools Enough to Close the Achievement Gap? Evidence from a Bold Social Experiment in Harlem
Will Dobbie and Roland G. Fryer, Jr.
Harlem Children’s Zone® (HCZ) is arguably the most ambitious social experiment to alleviate poverty of our time. We provide the first empirical test of the causal impact of HCZ on educational outcomes, with an eye toward informing the long-standing debate whether schools alone can eliminate the achievement gap or whether the issues that poor children bring to school are too much for educators to overcome. We implement two identification strategies. First, we exploit the fact that HCZ charter schools are required to select students by lottery when the demand for slots exceeds supply. Second, we use the interaction between a student’s home address and cohort year as an instrumental variable. Both approaches lead us to the same story: Harlem Children’s Zone is enormously effective at increasing the achievement of the poorest minority children. Taken at face value, the effects in middle school are enough to reverse the black-white achievement gap in mathematics and reduce it in English Language Arts. The effects in elementary school close the racial achievement gap in both subjects. Harlem Gems and The Baby College®, the only two community programs in HCZ that keep detailed administrative data, show mixed success. We conclude by presenting three pieces of evidence that high-quality schools or high-quality schools coupled with community investments generate the achievement gains. Community investments alone cannot explain the results.