When comparing the education systems of different countries,simply comparing test scores such as PISA or NAEP (or whatever) gives insufficient information to draw conclusions or set policy, but not for the reasons commonly cited. It is not that Norway has a smaller population, or that Japan's system is centrally managed, or whether a society is “homogenous,” or the tested population is comprised of “the best students.” The first step to comparing and evaluating different education systems is to understand the goals of the system.
For example, the goals of the Japanese education system are crystal clear and common knowledge. The main goal is for students to pass the university entrance examination. Passing does not mean exceeding some predetermined cut-off score. It means avoiding elimination. If there are openings in a particular university for, say, 3000 freshman, and 3050 apply, the test eliminates the lowest 50 applicants. The test score of the 51st applicant may be abysmally low, but it will still be a “passing” score. More typically, there may be 3000 slots, but 30,000 applicants. Passing scores will need to be very high, often around 95% correct on a test much harder than any SAT.
In America, ask a hundred people what the goals of the education system are, and it is like playing Family Feud. There is no real consensus. Society cannot decide what outcomes education should produce. Looking to Japan (or Norway or anywhere) as a model is unhelpful in such an environment. Nevertheless, we should ask whether the system is meeting its own stated goals. In America, with such nebulous goals, the question is hard to answer.
With Japan, the question is easy to state. Do Japanese students in academic high schools pass university entrance exams? First, we must understand there are three mutually exclusive high school systems. The students of two of the systems, vocational, and commercial, have no intention of taking university entrance exams. The students of the third system, the academic high school, are in the academic high schools precisely because they want to go to college. The only measure of learning that counts in Japan is the national university entrance examination.
By this measure, the Japanese marvel that American think the Japanese system is so great. It is true that when certain non-randomizing conditions are controlled, the Japanese still excel Americans on international studies of student achievement. By their own intra-country standards, they worry that so few students succeed. In a typical case, one year 309 students graduated from the Japanese academic high school where I worked. Of these 309 students only 93 thought themselves ready for the entrance exams and applied. Of those 93 only five passed. 304 students chose to continue studying for the entrance exams, attend a less-selective private university, or go to work. Of those who continued to study, only seventeen eventually passed the entrance exam. Thus 287 (or 93%) had to settle for less ambitious life goals.
The results indicate that even though Japanese students are learning, the Japanese education system fails to meet its express goals. The conundrum is that success would actually be a societal disaster. If entrance exams were a matter of exceeding a cut-off score, the universities would be swamped. Secondly, if all these university students graduated, they would expect to attain jobs commensurate with their education. Society does not have enough such jobs. No society does. No matter what the ideals, no society can absorb the success such ideals desire. Not Japan and not America and not anywhere. America's ideal of college for all is destined to disappoint.