Another Meaningless Statistical Analysis of School Performance

One great benefit of my having sat on a school board for the past 15 months is that it has forced me to dig into the details behind some of the bold statements made almost daily by well-meaning, but mathematically-challenged individuals who regularly tell us on what’s wrong with education in the US.  As a fortunate former student of Temple professor, John Allen Paulos, author of NY Times bestseller “Innumeracy”, wherein he laments the sad state of the media’s and many others’ ability to discern relevant fact from fiction with statistical analysis; I am often keen to check the facts and assumptions on various reports that claim some blinding realization after a new analysis of data.

Here’s just one more recent example of what I now consider to be complete and utter drivel as it proceeds to use a limited set of metrics, highly nuanced in their own right, to draw a value conclusion for the cost of public education.  It is a report by a group interested in science, technology, engineering, and mathematics or STEM education (of which I am an obvious fan), but sometimes even these well-meaning, and presumably well-educated folks fail to get things quite right.

So let me make this very clear and simple, because it is.  Quality and value in education is NOT measured just by performance on the core subjects of reading and math in relation to aggregated, district-level, per-student costs.  It is completely irresponsible to suggest otherwise, and we parents, and anyone who understands that a productive, innovative society contains more than just good readers and mathematicians; must speak out that we want our schools to be more than just mills for a limited set of core academic subjects.  We want good productive, global citizens and the best schools deliver that – my own being just one example.

Providing more, costs more.

It takes staffing, training, facilities, and planning to offer quality programs outside of the core subjects, but there are no national metrics that measure the quality of the students that result from such programs.  Whether it is through sports, the arts, advanced and elective subjects, or the wide array of extracurricular activities offered by wholesome communities, we know that students become better people through these programs and they are CHEAP to offer when they are included as part of a total academic program of study and local educational system.

We have to fight for the development of national comparative metrics to show that schools that offer more also deliver more.  While I do not have a specific list of such metrics, they would likely include a multitude of accreditation bodies in sports, the arts, etc. that evaluate the quality of the “beyond core” programs.  They almost certainly include long-term information gathering that follows students after they leave our schools with such considerations as continued college performance, job placement and salary statistics, award tracking, publishing statistics, and beyond.  Many of these items are not even being collected today, so this is a real uphill battle, but with modern technology, it is increasingly possible to do so without violating any privacy concerns.

So, what works and how do we know we’re getting our money’s worth?

The exact things that affect student achievement are often unclear, but through all of the research I have dug into, poverty, quality teacher training, and parent involvement are clear outliers for what works.  This is no surprise, which is probably why it receives almost no media attention.  The basic ideas are that students learn best when they are prepared to learn (fed, rested, and clothed), have a supportive environment at home, and have qualified instructors who inherently seek to collaborate and bring new approaches to teaching.  And we generate the best new citizens when we offer a broad array of intellectual experiences and training, which naturally implies a “beyond core” approach to total education.

I note here again the book and this summary article by Diane Ravitch who presents, with much greater credibility and detail, her experienced recommendations for a value system in public education.  These are not complicated ideas, but they do take focus and dedication, which makes them hard to do.

And for a reasonable example of statistical analysis, here’s one by a principal in Kansas showing how poverty is a significant factor in achievement and demonstrates a proper interpretation of  the recent PISA study (a study that, by the pundits, showed that the US was clearly lagging internationally).  This analysis shows if the US performance is considered against like levels of poverty, we are actually exceptional in our performance.  (Might this indeed be the reason that the US has lead the world economy and is forecast to do so for decades to come?)  We should be proud of the US public education system, contrary to the punditry.

The reason this analysis is more credible is that it is not trying to draw conclusions based on data that was not collected and there is only one, simple hypothesis being made, i.e., that there is a strong correlation between poverty and achievement.  While we must always be cautious about drawing causes from correlation, the intermediate links between poverty and academic performance are simply too easy to see.  Poverty negatively affects academic achievement, and we should fight its existence and effects on education.

Finally, lest you believe private or charter schools are somehow a standout here, please know that neither category of school is accountable to the public in the way that truly public schools are.  Private schools do not even report their academic testing metrics or their financial data.  While there are some strong performers just as in conventional public schools, in general, charter schools, and especially cyber-charter schools, provide little or no benefit in academic performance.

Producing outstanding young people requires more than just the core academic subjects, and good programs cost money.

In the end, our metrics need to be expanded to reflect all that we have and want in our public education system.


The Economic Policy Institute recently published this article that speaks directly to many of these issues.