The Education Train May Have Left the Station

The education train may have left the station, leaving traditional educators still groping their way blindly to the platform.

New, innovative, non-educators are poised to blaze the new education trails needed to help America reclaim its status as the world class education system. These new style educators usually have not earned education degrees, have not taught school, and do not possess any state teaching licenses. All that stuff is so last century. Nevertheless, they may be at the forefront of the education reform so many of us yearn for.

I am not talking about technology, a relatively recent buzzword near and dear to education writers and grant-funding foundations. A Request for Proposal without a technology piece is getting pretty rare. However, technology is often nothing more than gussied-up drill-and-kill.

The people currently studying the dynamics of learning are not necessarily publishing in journals, but they are implementing what they learn in domains outside of traditional education settings. Second Life just held its first commencement ceremony honoring virtual students who earned degrees entirely in the virtual world. Talk about online education. I just want to know if the real person behind the Second Life avatar has acquired real-life marketable skills. I mean if the avatar had to complete assignments and pass tests to earn a virtual degree, is it possible the person behind the avatar acquired that knowledge. And if not, could the concept be designed so that the puppetmaster, so to speak, does acquire the puppet's skills? Intriguing.

The government has commissioned Visual Purple to create training simulations that put participants in decision-making roles to maximize learning by doing. Robert Kiyosaki, of Rich Dad, Poor Dad fame, has a game, The CashFlow Game, out to help people master his strategies and tactics. The idea behind these simulations is the the puppet represents the puppetmaster, and the puppetmaster must learn and integrate knowledge and skills in order for the puppet to succeed. The acquired knowledge and skills is then directly useful in real life work.

So am I just talking about games and simulations? No. Games and simulations are nothing new. I used Reader Rabbit, Oregon Trail, Operation Frog, Sim Ant, and other programs with positive effects many, many years ago. But those programs were supplemental to my main teaching agenda, and compared with some of the new developments, frankly clumsy and primitive. With a well-designed curriculum, some of the new games can BE the teaching environment.

To be clear, I am NOT saying you have to subscribe to the opinions of any of the purveyors of games and simulations. I AM saying that Confucius was right. The amount remembered depends on the level of processing. The simulations seek to maximize learning at the active, 90% level.

Uncontrollable Variables Muddy Evaluations

Does Jonathan Alter at Newsweek really know what he wants? His recent article, Peanut-Butter Politics, rightly pinpoints teacher effectiveness as a crucial component of classroom effectiveness, but accuses the teachers union of reluctance to actually hold teachers accountable. The problem is he has no viable accountability plan except a nebulous call for measuring teacher effectiveness in the classroom.

Teacher effectiveness–say it three times. Last week a group called the New Teacher Project released a report titled "The Widget Effect" that argues that teachers are viewed as indistinguishable widgets–states and districts are "indifferent to variations in teacher performance"–and notes that more than 99 percent of teachers are rated satisfactory. The whole country is like Garrison Keillor's Lake Woebegon, except all the teachers are above average, too.

Why? The short answer is teachers' unions. Duncan complained recently that the California school system has a harmful "firewall" between student evaluation and teacher evaluation. In other words, teachers can't be evaluated on whether their students actually learned anything between September and June. The head of the San Francisco union says it's nuts to judge teachers on whether there's evidence that shows improvement in their classrooms. An A for accountability, eh?

...snip...

It takes a tough man to say, in the middle of a recession, "no improvement, no check." But if not now, when?

I addressed The Widget Effect a couple weeks ago. It is not so much that there is a lack of desire to hold teachers accountable. The main problem is that there are simply too many variables the teacher does not control. No one has yet proposed any fair way of evaluating teachers. And no teacher can or should be held responsible for (for example) the drunken uncle who lives in the student's home.

The worthlessness of evaluations creates a major disconnect in the school policy.

Though it is widely accepted that a teacher’s effectiveness matters more than any other school factor in student success or failure, it is almost never considered in critical decisions such as how teachers are hired, developed or retained.

Teacher effectiveness cannot be considered because teacher effectiveness is unknown. What's more, researchers have no consensus as to the characteristics of an effective teacher.
I would like to address the first two points.

It is easy to be negative and overlook the legions of highly motivated, highly competent, and highly effective teachers in our classrooms. In spite of the evaluation difficulties, we know they are there. here's the thing: many are recognized only years after a student has benefited from their influence. At the time, their students, with their lack of life experience, may not have realized what a treasure their teacher was. In fact, they may have even “hated” their teacher. Nevertheless, great teachers populate our classrooms in great numbers. A commonly appearing estimate is 50%. Around 50% of education students have the right stuff, but nearly all students will graduate and end up in our schools. Any college of education cohort can differentiate the more able from the less able among their peers. Maybe our colleges of education should be more selective, evaluating teaching candidates for suitability long before they have invested four plus years of time and money in becoming teachers.

Students (and Teachers) Do Not Understand Math

Three weeks ago educators wondered if conclusions from the Response to Intervention (RTI) research might be applicable to mathematics.

Educators gathered here last week to discuss a recent federal “practice guide” on response to intervention for students struggling in mathematics agreed that applying the RTI approach to that subject is challenging. But they also suggested that doing so was worth the effort.

It is instructive to discover the reasons applying the RTI approach to math is challenging.

Response to Intervention involves six steps: Screen, Teach, Intervene, Probe, Chart and Adjust.

Screen
Valid screening measures predict who is, and who is not, at risk for future reading difficulty. These measures are administered to determine if a child is at risk for failing a state's "high stakes" end of year achievement test, by which the state measures a school's overall performance. Children considered to be "at risk" are expected to experience difficulty responding (not keeping up) in the core curriculum as traditionally delivered in the regular general education classroom. Note: Due to the desire to capture all children who are truly "at risk," the false positive rate of early screening may be as high as 50 percent. In other words, as many as half of all the children who are identified as "at risk" by early screening may not be truly "at risk."
Teach
Core curriculum in the regular general education class should be research-based and field tested. This means, based on evidence from converging research, that the core curriculum contains all the elements found necessary to effectively teach reading and has a known track record of success. Such curriculum is to be delivered by "highly qualified" teachers sufficiently trained to deliver the selected instruction as intended, i.e., with fidelity to design. My note: Notice the language says, “trained to deliver the selected instruction as intended.” I knew one highly competent first-grade teacher in California that refused to deliver the selected instruction (whole language) as intended. It was a good thing because her students ALL learned to read even as California fell to 49th place in reading during the whole language period. The fad lasted until 1995 when phonics was reinstated in the curriculum. The change in the role of the teacher as indicated by the language I have noted is problematic.
Intervene
Provide "at risk" children with enhanced opportunities to learn, possibly including, but not limited to, additional time exposed to the core curriculum in small groups (3-6 students), other supplementary instruction, or special education.
Probe (progress monitoring)
Progress monitoring tests are brief measures of specific reading skills that are administered to determine if the child receiving intervention is responding as intended. They are given frequently, at least once every two weeks.
Chart
Progress is regularly charted to provide a visual record of actual rate of gain in specific reading skills in relation to a specified goal. The goal of intervention is for the child to improve relative standing and perform at or closer to grade level standards and is individualized according to the unique needs of the child.
Adjust
Depending on whether the child is achieving a rate of progress determined by his or her individualized goal, the manner and intensity of intervention will be adjusted. The cycle of progress-monitoring and adjustment of intervention will continue, even if a determination for special education eligibility is made.

Math educators met June 10, 2009 to explore whether the same six steps would be just as effective in math as in reading.

Educators at last week’s event said that fitting math into an RTI framework is hard, but that they believe it is now vital to improving math performance for struggling students.

One of the main difficulties is finding suitable math education materials.

Judith Russ, the mathematics curriculum supervisor for the 134,000-student Prince George’s County district in Maryland, said for her part that finding the right materials is hard.
The instructional materials “are not looking at building conceptual understanding. That’s one of the challenges we have,” she said.

The first step is to screen.

Karen D. Cheser, the assistant superintendent for learning support services for the 20,000-student Boone County district in Florence, Ky., said her school system started using RTI in reading two years ago, and had initially planned to leave math for later. But indications that students were becoming weaker as they reached higher-level math classes, among other factors, pushed the district to act.

Ms. Cheser said the district created its own universal screening program, which allows teachers to dig into what was going wrong for many students. It turned out that many students needed to focus so hard on computation that they were unable to grasp more sophisticated concepts, she said.

The panel found that children's gaps in mathematical understanding were fairly predictable.

... remediation for students in grades K-5 should focus on the properties of whole numbers, like counting, addition, and subtraction. Older students, up to 8th grade, should learn rational numbers in depth, including the meanings of ratios, decimals, and percentages, the panel recommends.

Another recommendation is that all students who need extra math assistance should work on fluent retrieval of basic arithmetic facts, like simple addition and multiplication. Higher-level mathematics often assumes that students can quickly recall facts like “3 times 9” or “11 minus 7,” when such operations may be difficult for those lagging behind their peers, the panel found.

The current reliance on the calculator does nothing to promote fluency with math facts. In fact, some teachers say that, with the ubiquitousness of calculators, it is no longer necessary to memorize math facts. But I know that students who cannot readily retrieve math facts struggle with algebra, and the research is confirming my observations.

Mathematics instruction has emphasized procedural competence over conceptual understanding. You do not necessarily have to understand the mathematics underlying long division as long as you can perform the operation. Sadly, students who perform mathematical operations reliably are told (through test grades) that they understand math when the reality is that they may have no idea why they do what they do. They do not “understand” math. The math materials in our schools “are not looking at building conceptual understanding,” as Judith Russ, mathematics curriculum supervisor, noted. But at least in times past, students were expected to memorize math facts in order to complete the procedural operations. So the situation is that students are not understanding math concepts, nor do they have the basic tools, math facts, for mechanically solving problems.

Years ago we used to call screening “diagnosis” and the best teachers have always made diagnosis part of their teaching practice. I used to diagnose struggling algebra students one-by-one. I still diagnose one-by-one, but nearly always I find the same gaps. They do not understand the function of place value (even if they can name a digit's place). They do not understand whole number properties (and fail to apply properties to numbers that do not look like “numbers,” especially numbers containing variables. They do not understand the difference between one and zero (such as when the “cancel” and say something like, “That's zippo.” They certainly do not understand fractions. Once I address these four abysses of knowledge, algebra suddenly becomes straightforward and even beautiful.

Is Quality Education a Lost Cause?

I love reading old stuff. Did some expert pundit's analysis and future projections come to pass or not? A quarter century ago, there were lots of articles wondering what we would do with all our future free time. The answer turned out to be---work even more. A quarter century ago, lots of articles wondered how America would solve its looming math and science teacher shortage. The answer turned out to be ---not much. Many universities are importing math and science professors from other countries because America is not producing its own. So it was interesting to reread an old article from Time entitled "Help! Teacher Can't Teach" dated Monday, Jun. 16, 1980, nearly thirty years ago. It could have been written yesterday.

Like some vast jury gradually and reluctantly arriving at a verdict, politicians, educators and especially millions of parents have come to believe that the U.S. public schools are in parlous trouble. ..Experts confirm that students today get at least 25% more As and Bs than they did 15 years ago, but know less.

Society holds the teachers responsible.

the new complaints about teachering also arise from a dismaying discovery: quite a few teachers (estimates range up to 20%) simply have not mastered the basic skills in reading, writing and arithmetic that they are supposed to teach.

Even as criticism abounds, Time (and all of us) recognize that 20% is quite a bit less than 100%.

Of course, among the 2.2 million teachers in the nation's public schools are hundreds of thousands of skilled and dedicated people who, despite immense problems, manage to produce the miraculous blend of care and discipline, energy, learning and imagination that good teaching requires. ...The best-educated and most selfless teachers are highly critical and deeply concerned about the decline in teaching standards and educational procedures. Their frustration is perhaps the strongest warning signal of all.

Testing tends to be the first line of defense. Many states began mandating teacher competency tests, only to find that far too many practicing teachers were unable to pass these tests. Lest one should think that teacher competency tests were perhaps too hard, most required math typically taught between the eighth and tenth grades, and English at corresponding levels. Any teacher, presumably all of them college graduates, should be able to pass easily. But they do not.

I was astonished to be the first one finished with the California Basic Educational Skills Test (CBEST). I put that test away in less than two hours. It was supposed to be a four-hour test. I once took the National Teachers Examination (NTE) in early childhood education cold, no study, review or preparation of any kind. I had been a secondary teacher for many years. I scored at the 86th percentile. I was not happy. My score was too good for someone like me who had taken a test outside of my field. According to the normative data on my score report, the vast majority of test takers were graduates of early childhood education programs. I did better than 86% of them. Not good.

Education Secretary Arne Duncan's support of merit pay for teachers, though framed as a way to pay teachers more, is really just a disguised way of saying if teachers taught better, our schools would be better, and so maybe more money would motivate teachers to teach better. One of the problems of merit pay is the unstated assumption that teachers are not already doing their best.

Okay, let's face the issue of teacher quality head on.

1.Teaching credentials are no assurance of teacher quality.
2.Schools do not hire the best qualified candidates, but the cheapest.
3.School of education attract students of lower academic ability than other academic departments.
4.Graduates of colleges of education must often take basic teacher competency tests many times before they pass.
5.Math teachers often do not possess a profound understanding of fundamental mathematics.

Some schools, like The Equity Project charter school in New York*, is determined to acquire high-quality teachers. They are offering $125,000 per year and the application process is a grueling four-step process. The charter school has created such a grueling process because the usual documentation, university degree with or without a state teaching credential, is worthless.

It did not used to be like this.

In 1900, when only 6% of U.S. children graduated from high school, secondary school teachers were looked up to as scholars of considerable learning.

Things were going swimmingly as high schools graduation rates steadily improved to a high of 70% by the 1960's. Sputnik was a huge surprise in 1957.

Almost overnight, it was perceived that American training was not competitive with that of the U.S.S.R. Public criticism and government funds began to converge on U.S. schools. By 1964, achievement scores in math and reading had risen to an alltime high.

Let's repeat that: Public criticism and government funds began to converge on US schools. Though only a child, I remember that time well. Society did not simply complain and moan; society demanded action and the government responded. The result, which directly benefited me, was that by 1964 achievement scores in math and reading had risen to an all time high. Only genuine achievement would do because society had a stake in knowing accurately if education was working. There was no interest in the statistical juggling so common now. Want SAT score improvement? In 1995, the College Board simply added 100 points** to everyone's score.

Over the last thirty years, there has been plenty of societal moaning and complaining, but no demand, no collective will. So society had the education system it wants. What did now Research Professor of Education at New York University Diane Ravitch say thirty years ago?

Diane Ravitch: "It is really putting things backward to say that if children feel good about themselves, then they will achieve. Instead, if children are learning and achieving, then they feel good about themselves."

Colleges of education are still teaching a backwards concept of self-esteem.

Although the driving motivation to beat the Soviets to the moon was not the noblest, my generation was the last beneficiary of America's once legendary education system.

Ever since the mid-1960s, the average achievement of high school graduates has gone steadily downhill.

...snip...

Many teachers have come to see themselves as casualties in a losing battle for learning and order in an indulgent age. Society does not support them, though it expects them to compensate in the classroom for racial prejudice, economic inequality and parental indifference.

In 1957 it was Sputnik. What will it take today for society to set aside complacency, ideological wrangling, or perpetuation of social status quo?


*The Equity Project's 4-stage application process

**To be fair, the purpose of recalibrating scores was not to artificially raise scores, but to realign the scores to the population so that a score of 500 would once again be average. According to the New York Times article,

The average verbal score today is 424; the average math score, 478.
So the College Board officials have decided to "recenter" the scale, changing it so the average student will once again get scores of 500 on the verbal and math tests...
In 1941, when the current norms were established for scoring the S.A.T., the world was a very different place. A small group of middle- and upper-class Americans attended college...

As colleges diversified in the 1960's, opening their doors to more poor and first-generation Americans, S.A.T. scores began a steady drop. By 1969, the average verbal score was 462; today, it is 424.

Teachers are Widgets

So President Obama wants to get rid of the bad teachers.

"We need to make sure our students have the teacher they need to be successful. That means states and school districts taking steps to move bad teachers out of the classroom. Let me be clear: if a teacher is given a chance but still does not improve, there is no excuse for that person to continue teaching. I reject a system that rewards failure and protects a person from its consequences. The stakes are too high," Obama said.

Nobody wants bad teachers. But exactly who are the bad teachers? How do we go about identifying them? Oh, I know. Let's take a look at the teacher evaluations. Surely the evaluations, many of them professionally designed, will point out the bad teachers. After the bad teachers are identified, all that remains is the battle with the teacher's union, whose mission is to make sure bad teachers teach until retirement. (snark alert).

Every school I have ever seen has a program (at least in the employee handbook) for annual evaluations. In my whole career, I have been evaluated just five times by supervisors (principal, vice principal, or department chair). And here is something else everyone knows: the evaluations are worthless. Now a new study by the New Teacher Project confirms what everyone knows. Entitled “The Widget Effect,” the study show that teachers are fungible.

The study illustrates that teacher evaluation systems reflect and codify the “Widget Effect”—the fallacy that all teachers are essentially interchangeable—in several major ways:

All teachers are rated good or great. Less than 1 percent of teachers receive unsatisfactory ratings, even in schools where students fail to meet basic academic standards, year after year.

Excellence goes unrecognized. When excellent ratings are the norm, truly exceptional teachers cannot be formally identified. Nor can they be compensated, promoted or retained.

Professional development is inadequate. Almost 3 in 4 teachers did not receive any specific feedback on improving their performance in their last evaluation.

Novice teachers are neglected. Low expectations for beginning teachers translate into benign neglect in the classroom and a toothless tenure process.

Poor performance goes unaddressed. Half of the districts studied have not dismissed a single tenured teacher for poor performance in the past five years.

The worthlessness of evaluations creates a major disconnect in the school policy.

Though it is widely accepted that a teacher’s effectiveness matters more than any other school factor in student success or failure, it is almost never considered in critical decisions such as how teachers are hired, developed or retained.

Teacher effectiveness cannot be considered because teacher effectiveness is unknown. What's more, researchers have no consensus as to the characteristics of an effective teacher.
I would like to address the first two points.

All teachers are rated good or great. And because all teachers are good or great, excellence goes unrecognized.
At best, evaluations are worthless. In many schools, an evaluation is a pro forma process, if it happens at all. The busy administrator visits the class for a few minutes, walks out and writes the glowing report.

At worst, the evaluation is a retaliatory or evidence-fabrication tool. I am reminded of the young elementary art teacher whose reputation for excellence was well-known by staff and parents alike. Teachers dropping off their class at her classroom often lingered and teachers retrieving their class often came early to observe and hopefully glean some useful tips. One fine April morning the vice principal came to observe a class period and stayed for the whole class. His one and only comment after the class left: he did not like that the students were allowed to chat with their neighbors as they worked on their art.

The resulting evaluation was a disaster. On a 5-point scale, her average came to 2.7. She objected to the principal and he allowed her to write a rebuttal. But the rebuttal went nowhere. All that survived of the evaluation was the average which appeared on a list of all the teachers with all their 4.X averages. The school submitted the list to the district office.

She complained bitterly to the principal who told her not to worry—it would have no effect on her future career. She complained to her colleagues, some of whom interceded for her with the principal. You see, this young teacher had rebuffed the vice principal's advances at the school Christmas party. Her colleagues suggested the principal replace the vice principal's evaluation with one of his own, but he refused, saying it would be unseemly to override the vice-principal.

This story is not a fluke. Evaluations, if done at all, are often undertaken only because the teacher has entered the administrator's radar for some reason. In such situations, greatness cannot help but go unrecognized. I would go so far as to say that Teachers of the Year are not necessarily the top teachers. They are teachers with spare time. Many of the best teachers are simply too busy to fulfill the onerous essay and video requirements to be considered for a Teacher of the Year award.

What, you say. You thought Teachers of the Year were nominated for doing their jobs every day. Most Teachers of the Year are self-nominated. Typical is the application for the Arizona Teacher of the Year.

Teacher nominees/applicants must submit a written application that is reviewed by a panel of judges consisting of educators, students and members of the business community. Ten finalists are selected from the written applications.

The 10 finalists are asked to prepare a 15-minute videotape. The final selection process includes review of the videotape, an interview and an impromptu speech by each of the 10 finalists. Following that process, the Teacher of the Year is selected along with four “Ambassadors for Excellence” and five finalists. The Teacher of the Year and Ambassadors have multiple opportunities during the year to make public appearances throughout the state, speaking to professional, civic, educational, parent and student groups. (my bold)

The written application includes 13 double-spaced pages of essay material:

Educational History and Professional Development Activities (2 double-spaced typed pages) – 5 points
Professional Biography (2 double-spaced typed pages) – 5 points
Community Involvement (1 double-spaced typed page) – 5 points
Philosophy of Teaching (2 double-spaced typed pages) – 15 points.
Education Issues and Trends (2 double-spaced typed pages) – 15 points
The Teaching Profession (2 double-spaced typed pages) – 20 points
National Teacher of the Year Message (1 double-spaced typed page) – 10 points
Arizona Teacher of the Year Message (1 double-spaced typed page) – 10 points

The school administrator must agree in writing to approve up to 30 days for a substitute teacher to allow for newly-minted teacher of the year public appearances.


You may be the greatest teacher in the world, but if you are, and you are honest, you are very likely not going to write essays that will get you selected as teacher of the year, especially when you write about “education issues and trends” and “the teaching profession.” John Taylor Gatto was voted New York's Teacher of the Year in 1991 and immediately, with his acceptance speech, began telling everyone about the insidious goal of compulsory education to de-educate students. He has written several books, all with the same message. Somehow I have trouble believing the message he proclaimed from the Teacher of the Year platform was the same message he told the committee he would proclaim when he wrote his application essays.

Neither the complicated, multifaceted, self-selected Teacher of the Year evaluations nor the run-of-the-mill annual evaluations performed (or not) in most schools succeed in any meaningful way.

Can the Top-Scoring State Beat International Scores?

How does the math covered in the highest-ranking American state stack up against that of a top-scoring international performer?

International comparison studies typically focus on the comparing the scores achieved by same-age students in different countries. Also typically, students from Asian countries tend to outperform US students over and over again. Each time a report like that comes out, just as predictably there will be an out-pouring of the same old tired excuses. Their students are different from our students. Their culture is homogeneous whereas ours is diverse. Their schools are allowed to teach whereas our schools must meet social, medical and nutritional needs. Their parents value education whereas our parents, not so much. On and on. The excuses act as a sedative to put society back to sleep. Okay, society says, there are understandable reasons for the differences in performance. The results are not really comparable. Apples to oranges. What a relief. So we stop thinking about it.

Could there be something more?

Sean Cavanaugh of Edweekreports:

A host of recent studies have examined how U.S. students’ mathematics skills compare against those of their foreign peers. Now, a new analysis probes a more precise question: How does the math covered in the highest-ranking American state stack up against that of a top-scoring international performer?

Let's repeat the question: How does the math covered in the highest-ranking American state stack up against that of a top-scoring international performer? It does not matter whether the results are comparable or not. No matter the reason our kids come out second rate, other kids are beating our kids in the worldwide competition. Remember, President Obama said that if we want our kids to out-compete the world, we must out-educate them.

So how does the math covered stack up?

A study released last week finds that elementary students in Hong Kong are exposed to more difficult and complex math than pupils in Massachusetts, an elite scorer on national and international exams. The analysis, published by the American Institutes for Research, in Washington, examines the math content of Hong Kong and Massachusetts by comparing the two jurisdictions’ standardized tests in 3rd grade math.

We're talking about third grade, part of the foundation of the rest of a child's academic career. The study did not look at scores on a specially designed test for international comparison purposes. The study did not look at the content of such a specially designed test. The study examined the respective jurisdiction's in-house test, the standardized test for Massachusetts and Hong Kong. Even more interesting, the study had no interest in the children's scores on these tests. The study studied the test content itself. And why Massachusetts?

Massachusetts is also a consistent elite-scorer on the primary U.S. domestic test, the National Assessment of Educational Progress.

What the study found is that the Hong Kong test emphasizes number and measurement concepts. The test also contains a larger percentage of constructed responses rather than chosen responses. The Hong Kong test questions were more complex, requiring the application of knowledge and non-routine, multi-step solutions over simple recall. From the foundations, children in Hong Kong are tested on higher-order thinking skills than American children, even “elite” American children.

Do Chinese teachers teach to the test?

(Steven Leinwand, one of the study's authors), said the authors chose to examine test content in Hong Kong and Massachusetts because the two jurisdictions' early-grades math curricula were relatively similar—and because state tests in the United States tend to guide math instruction.

American educators “pay attention to the tests,” he observed. “If you change the state tests, it’s a powerful lever for what goes on in the classroom.”

In the US, the favorite quick and dirty way to reform education is to redesign the tests. That's what Arizona did in the 1990's with their AIMS test. Arizona created high-stakes tests for fifth, eighth and eleventh grade, as if new tests automatically change educational philosophy and encourage innovation. Even honor students flunked these tests. The overwhelming response to high-stakes tests is to teach to the test, a response well-documented by No Child Left Behind. When a test reflects existing educational philosophy, there is no need for sample tests or practice materials.


Liping Ma has documented the emphasis Chinese teachers place on concept development over computational procedures. James Stigler reiterated many of the same points. Chinese math education, exemplified by Hong Kong, already valued conceptual understanding and the test reflects that value. The US, regardless of all the pretty talk in the media, values computational procedures and the Massachusetts test reflects that value.

How did Mr. Leinwand put it? “... state tests in the United States tend to guide math instruction.” That is the large part of the problem. We are suppose to test what we teach, not teach what we test. The US mistakenly thinks testing drives instruction.

The Uncomfortable Conclusion

Laying solid foundations in the early years matters.

Hong Kong’s use of more difficult and complex test items could be connected to a higher proportion of its test-takers, 40 percent, scoring at the “advanced” TIMSS level, than Massachusetts, at 22 percent. Just 10 percent of American students, on the whole, reached that level, the authors argue. In addition, research shows a “strong correlation” between nations’ math performance in early and later grades, they say.

Why Standardized Testing Will Always Fail

The most basic characteristic of any test is validity, that is, whether the test actually tests what it purports to test. Everyone, from the “professionals” who write standardized tests, to the everyday classroom teacher putting together a five-point quiz, learns that a test that does not actually test what it claims to test is worthless. They all learned about validity in the colleges of education.

So John Pearson makes a great point when he observes that every test is a reading test.

TAKS is stressful enough to prepare for at the 3rd grade level, and our kids at least can get reading assistance on the math test! There has been a little bit of debate over exactly what that means, but at least it is specified that, on an individual basis, a student may ask to have a word or a question read aloud. This helps immensely, especially with a child who is a struggling reader and/or an English Language Learner.

However, after 3rd grade, the kids are completely on their own for every TAKS test -- excepting those kids with special modifications, of course. The vast majority of kids taking these tests every year cannot ask to have a word read, cannot ask for clarification on a question, cannot ask ANYTHING except a question about the directions, and the directions are usually "Pick the best answer."

So what it comes down to is that these kids are taking a series of reading tests. Some of them are ABOUT math or ABOUT science, but they don't strictly assess those subject areas as much as they assess whether or not the child can read the questions, some of which are highly complicated.

I knew a little boy in Japan who was completely bilingual in both Japanese and English, but who had attended only Japanese preschool and kindergarten. The first thing to understand about his situation is that the Japanese kindergarten ends near the end of March, so when he “graduated” from kindergarten, his parents decided to enroll him in an international school where instruction was conducted in English. The principal said the first grade teacher needed to access the boy's readiness.

On the appointed day in March, this boy sat down with a clearly unhappy first grade teacher. She did not want any new students entering her class so close to the end of the school year, especially one whose parents had the idea the child would go on to second grade after less than three months in first grade. The teacher asked a number of questions about fairy tales and a few addition problems and announced that the boy was “marginal.” She would allow him into her first grade class on the condition that the parents understood that in September he would very likely have to “repeat” the first grade. The parents accepted the condition.

In April, the school gave the annual Stanford 9 bubble tests. The first grade teacher made a copy of this boy's answer sheets to hand grade, because the score reports would not be available before the end of the school year. She needed ammunition for the parent-teacher conference she was sure she would need when she planned to tell the parents that yes, indeed, their son would have to repeat first grade.

To her utter astonishment, the boy had almost a perfect set of answer sheets. The score report, when it eventually arrived, placed the boy in the 99th percentile on every battery. Obviously he went to the second grade along with his class. Eventually the same boy graduated from an American university at age eighteen with a degree in chemistry.

So why did the teacher consider the boy marginal? Mostly because he did not know who Rumpleskilskin was. The boy could have told her all about Momotaro, a Japanese fairy tale character the teacher had never heard of, if only she had known to ask, except Momotaro was not included in the school's first grade curriculum anyway.

Imagine going to live in Russia for a year and taking a math class. After 3 months, you are given a math test in Russian, consisting of word problems and lengthy questions. I don't know about you, but I would fail that test miserably. Would ANYONE in their right mind think that that means I don't know math?? Or that that test accurately gauged my knowledge??

I was a teacher in that international school in Japan. I taught math and science to the middle-schoolers. Every year fully 50% of my students were non-native speakers of English. One year four of my students were non-English speakers who had transferred from the Japanese school just that year. Lucky for me I also speak Japanese. I was the only American teacher in the school who spoke Japanese. There were a few Japanese-speaking teacher's aides.

I made all kinds of accommodations to help my non-native English-speaking students. I paired each one with a native speaker for labs. I translated my instruction to Japanese on the fly on a regular basis. I adapted reading instruction techniques usually used in much lower grades to the science book as if the science book were a basal reader. I read words or whole questions from my tests for any student who asked. And for those four non-English speakers, I translated the whole test to Japanese. I did all these things because I knew what every tester should know, that is, the purpose of the tests. The purpose of my tests was to evaluate the student's mastery of my instruction with the corollary purpose of giving the students the best chance for success.

We may think the purpose of standardized is to evaluate individual student's knowledge, but in reality, the tests serve to rank students compared to the norming population, and then by extension, to rank the quality of the school relative to the norming population. The reality will always frustrate because the nature of norming means that half will be above the 50th and half will be below the 50th percentile when compared to the total population.

If some schools can attract an overabundance of topside students, obviously other schools will end up with an overabundance of bottomside students. Testing can, by design or not, perpetuate the inequality of educational opportunity and undermine any promising efforts of school reform.

So who would want to perpetuate inequality of educational opportunity? Sadly, dear parents and other adults, Lake Wobegone does not exist.