Can the Top-Scoring State Beat International Scores?

How does the math covered in the highest-ranking American state stack up against that of a top-scoring international performer?

International comparison studies typically focus on the comparing the scores achieved by same-age students in different countries. Also typically, students from Asian countries tend to outperform US students over and over again. Each time a report like that comes out, just as predictably there will be an out-pouring of the same old tired excuses. Their students are different from our students. Their culture is homogeneous whereas ours is diverse. Their schools are allowed to teach whereas our schools must meet social, medical and nutritional needs. Their parents value education whereas our parents, not so much. On and on. The excuses act as a sedative to put society back to sleep. Okay, society says, there are understandable reasons for the differences in performance. The results are not really comparable. Apples to oranges. What a relief. So we stop thinking about it.

Could there be something more?

Sean Cavanaugh of Edweekreports:

A host of recent studies have examined how U.S. students’ mathematics skills compare against those of their foreign peers. Now, a new analysis probes a more precise question: How does the math covered in the highest-ranking American state stack up against that of a top-scoring international performer?

Let's repeat the question: How does the math covered in the highest-ranking American state stack up against that of a top-scoring international performer? It does not matter whether the results are comparable or not. No matter the reason our kids come out second rate, other kids are beating our kids in the worldwide competition. Remember, President Obama said that if we want our kids to out-compete the world, we must out-educate them.

So how does the math covered stack up?

A study released last week finds that elementary students in Hong Kong are exposed to more difficult and complex math than pupils in Massachusetts, an elite scorer on national and international exams. The analysis, published by the American Institutes for Research, in Washington, examines the math content of Hong Kong and Massachusetts by comparing the two jurisdictions’ standardized tests in 3rd grade math.

We're talking about third grade, part of the foundation of the rest of a child's academic career. The study did not look at scores on a specially designed test for international comparison purposes. The study did not look at the content of such a specially designed test. The study examined the respective jurisdiction's in-house test, the standardized test for Massachusetts and Hong Kong. Even more interesting, the study had no interest in the children's scores on these tests. The study studied the test content itself. And why Massachusetts?

Massachusetts is also a consistent elite-scorer on the primary U.S. domestic test, the National Assessment of Educational Progress.

What the study found is that the Hong Kong test emphasizes number and measurement concepts. The test also contains a larger percentage of constructed responses rather than chosen responses. The Hong Kong test questions were more complex, requiring the application of knowledge and non-routine, multi-step solutions over simple recall. From the foundations, children in Hong Kong are tested on higher-order thinking skills than American children, even “elite” American children.

Do Chinese teachers teach to the test?

(Steven Leinwand, one of the study's authors), said the authors chose to examine test content in Hong Kong and Massachusetts because the two jurisdictions' early-grades math curricula were relatively similar—and because state tests in the United States tend to guide math instruction.

American educators “pay attention to the tests,” he observed. “If you change the state tests, it’s a powerful lever for what goes on in the classroom.”

In the US, the favorite quick and dirty way to reform education is to redesign the tests. That's what Arizona did in the 1990's with their AIMS test. Arizona created high-stakes tests for fifth, eighth and eleventh grade, as if new tests automatically change educational philosophy and encourage innovation. Even honor students flunked these tests. The overwhelming response to high-stakes tests is to teach to the test, a response well-documented by No Child Left Behind. When a test reflects existing educational philosophy, there is no need for sample tests or practice materials.


Liping Ma has documented the emphasis Chinese teachers place on concept development over computational procedures. James Stigler reiterated many of the same points. Chinese math education, exemplified by Hong Kong, already valued conceptual understanding and the test reflects that value. The US, regardless of all the pretty talk in the media, values computational procedures and the Massachusetts test reflects that value.

How did Mr. Leinwand put it? “... state tests in the United States tend to guide math instruction.” That is the large part of the problem. We are suppose to test what we teach, not teach what we test. The US mistakenly thinks testing drives instruction.

The Uncomfortable Conclusion

Laying solid foundations in the early years matters.

Hong Kong’s use of more difficult and complex test items could be connected to a higher proportion of its test-takers, 40 percent, scoring at the “advanced” TIMSS level, than Massachusetts, at 22 percent. Just 10 percent of American students, on the whole, reached that level, the authors argue. In addition, research shows a “strong correlation” between nations’ math performance in early and later grades, they say.