BEYOND THE HYPE: THE CREDO STUDY AND NJ CHARTER SCHOOL PERFORMANCE

In New Jersey, the CREDO charter school study released in late June was met with scant analysis and undue praise for improvements in charter school performance. The national take-away was much more sobering: the vast majority of charters perform at similar levels to, or worse than, district public schools. What’s more, the average learning gains for charter students compared to their public school peers are so small that the study raises important questions about the efficacy of charters as a means to improve educational outcomes.

The latest results published by CREDO (Center for Research on Education Outcomes at Stanford University) showed that NJ charters perform only negligibly better than traditional public schools. Predictably, NJ Education Commissioner Chris Cerf – a proponent of charter school expansion –applauded these results. Commissioner Cerf, however, conveniently ignored the results of a prior, more discrete CREDO analysis showing some success among charters in Newark, while those in other major NJ cities performed poorly in reading and showed no gains in math relative to district schools.

Behind the media hype, the CREDO study fails to recognize the almost negligible impact of charters on student learning. The study also suffers from methodological limitations – it does not properly address the selection bias of clustering more relatively advantaged students in charter classrooms and schools – that call into question its usefulness. Below we analyze three main problems with the study that cast serious doubts on CREDO’s claims of superior charter performance.

Crude Measures of Student Characteristics

Students in NJ district and charter schools differ in important ways that are glossed over when using broad classifications of student characteristics. In Newark, for example, charter schools serve fewer free-lunch eligible – or very poor –students and enroll twice as many students who qualify for reduced-priced lunch (family income up to 185% of the poverty level). In addition, the vast majority of special education students in charters fall into relatively mild classifications of speech or language impairment or “specific learning disability,” while the Newark Public Schools serve a much greater proportion of students with severe disabilities, such as autism or mental retardation.

The CREDO study ignores these important differences. The study attempts to control for student demographic differences by comparing academic growth among “virtual twins.” But some of these measures are so broadly defined that students assigned the same status in the report may actually be quite different. For example, CREDO does not distinguish among levels of disability, so a student with a mild learning disability is defined as a “twin” of a student with a more severe cognitive disability. Similarly, the “economic status” indicator combines free or reduced price lunch eligibility, lumping together all students whose level of family income is less than $40,000 a year. Because of the ways that students are sorted between district and charter schools, it is more likely that a less disadvantaged charter student is being matched with a more disadvantaged traditional public school student in the CREDO study. In this context, it would hardly be surprising if “virtual” charter students are found to make greater gains than their district school “twins.”

School and Classroom Effects Matter

The context of a student’s school and classroom affects his or her ability to learn; researchers call this the “peer effect.” If a student is in a classroom with other children who are more economically disadvantaged, are learning English, or have more severe disabilities, this can create a more challenging environment for learning. In New Jersey, less disadvantaged students tend to cluster in charters, while the most disadvantaged remain in district schools. The CREDO study does not control for the disproportionate sorting of students between charter and traditional public schools and the resulting peer effects. It is likely that some of the small learning gains experienced by charters are a result of this sorting alone.  

Close Schools, Improve Performance?

The CREDO study found that the charter sector improved only slightly in the past few years, and the vast majority of charter schools do not perform better than district schools. The study also found that charters as a whole showed no absolute learning gains, but that the sector improved simply as a result of closing low-performing charter schools. In other words, the performance of existing charters did not improve over time, new charters actually perform worse than existing charters, and overall performance increased only because some charter schools were closed. The study also ignored the potential learning losses suffered by children in charter schools that close.  

Most importantly, the CREDO study offers no explanation of or rationale for the methods and practices that result in positive outcomes for the few charter schools that do outperform traditional schools. The study does not provide any evidence that successful models can be copied, or that low-performing charters can improve. The strategy of closing schools may make the sector look better on paper, but it does nothing to increase the number of students receiving a quality education. In fact, it introduces more disruption for students who already are struggling due to an ineffective educational environment. Focusing on closing bad schools as the “strongest available tool to improve overall sector quality” is an expensive and extremely disruptive policy.

Bottom line: be wary of the hype surrounding the CREDO study.  Not only are there significant limits to the study’s methodological approach, but its policy recommendations also are out of line with its findings. In truth, most charters are not performing any better than district schools and are not improving outcomes over time. The small gains that do exist have no appreciable impact on narrowing existing achievement gaps and raising student performance. The expense and disruption of a continuous cycle of opening and closing schools is an expensive experiment with questionable outcomes.  

 

Press Contact:

Sharon Krengel
Policy and Outreach Director
skrengel@edlawcenter.org
973-624-1815, x 24

Press Contact:
Sharon Krengel
Director of Policy, Strategic Partnerships and Communications
skrengel@edlawcenter.org
973-624-1815, x240