ORIGINAL RESEARCH

Beating the odds, or beating the test?

High performing Charter Schools

Rob Levine
September 4, 2012

“Odds-beating charter school. " Those words are like an impenetrable shield for those who operate such places. They are also the holy grail of the education reform movement, which is constantly seeking shortcuts to radically increase measures of educational achievement, which these days is pretty much defined by increased math and language test scores.

One problem with radical test score gains, as many researchers have noted, is that miraculous improvements in test scores over short periods of time are more often the result of cheating, student skimming, or other test manipulation. We’ve seen this pattern repeated all over the nation, starting with the so-called Texas Miracle under former US education secretary Rod Paige’s oversight.

Something very similar happened in New York state, where unrevealed relaxations in state testing standards led to a multi-year belief in the efficacy of reforms there. When results from the National Assessment of Educational Progress (NAEP) tests revealed the gains were illusory, New York admitted the test changes, thereby wiping out half a decade of supposed achievement gains on math and language tests.

Real improvements in test scores are more likely the result of longer-term changes that cause slow but steady rises in achievement. In fact, test results from decades of the NAEP show just such changes for the country as a whole. Scores for all US students have gone up by a full academic year over the past few decades, including breakout scores for various demographic groups including blacks and whites, and achievement gaps have actually been declining. Ironically (or, not) these gains have actually been slowing in the age of No Child Left Behind.

In Minnesota, birthplace of the charter school movement, one charter school operator labeled an “odds beater " has put a new twist on the concept. Eric Mahmoud, a former engineer and convicted mortgage fraudster, operates a group of segregated charter schools targeted at black, inner-city poor children that employs longer school days and years, strict discipline, and an unabashed strategy of beating state achievement tests.

For his success in raising math and reading scores, Mahmoud has been given rock-star status among charter proponents and recently was inducted into the National Alliance for Public Charter Schools hall of fame. In return, the Minneapolis school board recently awarded him the right to open four more charter schools sponsored by the city’s school district, all presumably to be run in what a local conservative columnist laudingly called his “drill and kill " style. News stories and opinion pieces from traditional to the alternative press have nonetheless sung his praises, so desperate is the reform community to raise test scores. Meanwhile, almost no one has questioned the efficacy or morality of deliberately segregated schools, nor ones so focused on improving specific test scores. It seemed that, until very recently, Mahmoud could do no wrong.

Three weeks ago that might all have changed with a series of investigative stories in the Minneapolis Star Tribune by veteran reporter Steve Brandt highlighting fiscal and management questions that, given the lax controls on charter schools and Mahmoud’s previous mortgage crime are unsurprising. Among Brandt’s cumulative reporting is the following:

• Mahmoud’s combined salary for his duties running charter schools (which total 900 enrollment) and their parent organization amounted to $273,000, more than any other superintendent in the state. In addition his wife, whose occupation is listed as “secretary, " took home $105,000, and his daughter got $28,000 as an “administrative aide. "
• Three of the four board members of the schools’ parent organization, called “Seed Daycare, " which leases property to the school and provides administrative services, are Mahmoud family members.
• A letter from the charter school’s authorizer “asserts that teachers whom Harvest (one of Mahmoud’s charter schools) paid as contractors clearly fall under the IRS definition of employees. "
• The school’s authorizer, Audubon Center of the North Woods, sent a “letter of deficiency " to Mahmoud asking for clarification of the organization’s finances and for an explanation of “frequent payments among Mahmoud’s organizations and Mahmoud himself, " and that “it’s unclear what they are for and whether state money that supports Harvest and related entities is being used appropriately. "
• The Minnesota Department of Education is seeking to recoup money it granted to Mahmoud for an earlier, failed charter school used for its construction and to equip it. The MNDOE turned this issue over to the state auditor.
• Mahmoud, who operates deliberately segregated schools, is claiming racial discrimination, a claim disputed by the state, in the revocation of a $300,000 grant to one of his schools. The grant was to be used for a federally funded program called Reading First, which required the school to hire a full-time literacy coordinator and to use licensed teachers in the program, neither of which Mahmoud did.

But more important than the financial and management problems at Seed schools is the nature and structure of the educational pedagogy. The schools teach to the tests. Not just teach to them, but engineer high test scores with frequent testing, 100 minute blocks of reading and math each day and other methods. Few local writers or pundits pause to understand the implications of a strategy so strictly focused on state tests. Alleen Brown quoted Mahmoud’s thoughts on teaching to the test for an article in the TC Daily Planet (emphasis added):

“If the standards were different, content absolutely would look different, " said director Mahmoud. He dismissed a question about whether designing instruction after standards amounts to teaching to the test, saying that if an assessment is aligned to the standards, then there shouldn’t be a problem. “The worst thing in the world is to teach to something other than what the students are going to be assessed on. "

That is the discussion we should be having. Do we want to turn over even more of our black students to authoritarian schools that “drill and kill "? The tests themselves, which actually test a very narrow range of skills, are still sometimes useful for what they reveal. Below is a common chart about Mahmoud’s schools marvelous achievement rates:

From the chart you can tell that Harvest and Best both had superior proficiency rates and that Harvest had truly outstanding results. Education reformers who tout the primacy of these test score measures say the focus on math and reading is necessary to begin the learning pyramid. If that were true you’d expect Mahmoud’s students to do better in science, too, since math is an important component. In fact, as the charts from the Minnesota Department of Education below show, Harvest and Best students are doing poorly and getting worse on science tests. The first chart compares Harvest fifth grade science scores with those of all Minnesota students:

An average of all Minnesota students for all grades is a science proficiency rate of approximately 45%. Harvest Prep has a proficiency approaching zero. Next are the science scores for the Best Academy:

Here the story is much the same. Science scores for Best Academy fifth graders are bad and trending worse. In fairness, Best has improved science scores for its eighth grade students, who have improved to about a 30 percent proficiency rate. Best and Harvest science scores for fifth graders are actually much worse than statewide scores for students who are both black and receive free or reduced lunch, where scores are approaching 20 percent proficiency:

The perils of our obsession with narrow testing and the incentives for charter schools to teach to the test were spelled out last year by Michael Diedrich, a then Policy Associate for the think tank Minnesota 2020:

“..the data "¦ support the hypothesis that what isn’t tested isn’t taught (or at least not with the same intensity).

We need to remind ourselves what the goal of our education system is. It’s not just about scoring well on the tests we have. It’s about comprehensive student readiness for post-secondary success. To punish and reward schools using just one and a half subjects (reading comprehension being only one part of English/Language Arts) is to encourage a narrow-minded focus on low-level testable skills. The MCAs disproportionately assess students at the lowest levels of learning; the skills necessary for critical thinking, effective creation, and original problem-solving don’t make the cut.

Even the reading tests being used in Minnesota for students before the ninth grade have severe deficiencies that further constricts what’s taught to early learners, as Diedrich pointed out in a comment to his piece:

“ "¦there are plenty of math and reading skills that aren’t tested by the MCA. The vast majority of MCA questions come from the bottom two levels of Bloom’s Taxonomy (“remembering " and “understanding "), a few come from the middle (“applying, " “analyzing, " and “evaluating ") and none at all come from the top (“creating "). Because higher level skills aren’t tested, they don’t get taught, yet these are the skills necessary for students’ future success. "

Education is not the same as beating the test

The problems Minnesota has seen with “test beating " schools like those run by Eric Mahmoud — financial and management irregularities, forms of self-dealing that enrich operators, and engineering test results that bear little resemblance to forms of deep and wide learning — are foreseeable given the relaxed rules that apply to charters and our obsession with test results from a narrow range of subjects.

Mahmoud does not come from a teaching background — he was an engineer, a profession that looks at a problem domain, assesses the metrics on which he or she will be judged and finds a way to meet those benchmarks. From this perspective it is easy to see why his schools operate in a fashion to beat state tests — those are virtually the only metrics that matter, as Mahmoud himself has admitted. But this only shows how an unhealthy emphasis on testing has perverted the educational process.

In 1976 social science researcher Donald Campbell coined his “Campbell’s Law " hypothesis, which succinctly explains why our unhealthy obsession with standardized testing is leading to educational failure:

“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor. "

Campbell explained his law in a way that directly anticipated Mahmoud’s teaching strategy:

“ "¦achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways. "

No one can fault the parents of black children in Minneapolis for wanting their kids to excel in math and reading. But methods matter and children aren’t widgets to be produced in an industrial process. We shouldn’t be surprised that Mahmoud manages to increase math and reading scores given the longer school year, longer school days, a singular focus on beating test scores, and the self-selected nature of his charter school student population. It would be more surprising if he wasn’t able to raise those test scores. But those marks come with a high price to a fully rounded education. Beating tests must not be confused with beating the odds. On a national level President Obama’s educational policies encompassed in his Race To The Top program only make these problems worse with their focus on expanding charter schools and more standardized testing. We can do better.