Does teacher professional development work?

Apparently not. So concludes a pair of studies, one focused on middle school mathematics, and the other on early reading.

Results after one year of providing teachers math professional development (PD) indicate no improvement on their students' math achievement when compared to teachers who did not receive the study-provided PD.

The result is surprising and disconcerting. The professional development provided by the study emphasized fractions, decimals, percents, and ratios. Education students self-report these very topics as their number one weaknesses. Nevertheless, education students are sure to graduate and enter the classroom with their weaknesses intact, even after completing the mandated mathematics coursework.

The early reading study came to a similar conclusion.

Although there were positive impacts on teacher’s knowledge of scientifically based reading instruction and on one of the three instructional practices promoted by the study PD, neither PD intervention resulted in significantly higher student test scores at the end of the one-year treatment.

Eric A. Hanushek, from Stanford University, concluded, “...you can't change ineffective teachers into effective ones.” Hilda Borko, an education professor also from Stanford, counters with, “We know teacher change takes time.”

The truth is probably a blend of both assertions. A competent, reflective teacher is quick to adopt useful new skills, approaches, or tactics. The discovery of well-designed materials and curriculum induces nearly instantaneous change. Effective teachers are always looking for new tools to add to their every growing collection. If possible, the teacher will roll out the new tool the very next day. If the teacher finds the the new tool requires a major or even minor redesign of lesson plans, materials, and handouts, well, so much for summer vacation travel plans. Being the best that they can be is a core value of effective teachers.

Both studies tested for student improvement after one year and found none. But since both studies also found that the professional development did not increase teacher math knowledge or alter teacher practices, of course there would be no gains in student test scores.

The studies noted that participating teachers were “engaged” in the professional development, thus ruling out lack of engagement as a reason for the disappointing lack of results. However, as all teachers know, teachers can fully participate in professional development workshops, and still walk out thinking, rightly or wrongly, “Well, that was a waste of time.” Engagement is one of those necessary, but not sufficient conditions.

If we agree that effective teachers will readily adopt recognizably useful skills and concepts, why is professional development looking to be yet another waste of everyone's time, effort and money? Could it be that the schools of education need to be more selective in admitting education students? As it is, once a student is admitted, a high GPA at least in education courses and graduation is all but assured. Many states grant automatic teaching credentials to graduates of state education programs. Yet the National Bureau of Economic Research (NBER) reports that credentials make no difference.

One of the first things researchers did with the computed teachers' effects was investigate whether they were closely related to the teacher credentials upon which achievement was traditionally regressed. The answer was generally no: credentials do not explain teacher effects for the most part...differences in certification explain only a small fraction (if any) of the variation in achievement: differences among teachers with the same certification dwarf the differences associated with certification (or the lack thereof).

A study from New York found :

This evidence suggests that classroom performance during the first two years, rather than certification status, is a more reliable indicator of a teacher's future effectiveness.

Schools and universities continue to hype, and the public continues to buy, credentialing as some indication of quality.


Could it mean that society needs to demand more of its schools of education, and that in the short term, universities must be willing to forego the tuition cash cow the schools of education represent? I know that whenever I have suggested tougher requirements for student admission, the universities invariably respond that they cannot afford the loss of tuition. Thus the vicious circle never ends.

“We must break the cycle in which we find ourselves,” said William H. Schmidt...“A weak K-12 mathematics curriculum in the U.S., taught by teachers with an inadequate mathematics background, produces high school graduates who are at a disadvantage. When some of these students become future teachers and are not given a strong background in mathematics during teacher preparation, the cycle continues,”

Professor Schmidt was referring to the results of an international study on the preparedness of mathematics teachers, a study he supervised. Research is showing that when entry level pay for teachers is raised, the quality of education applicants increases. Perhaps society needs to agree to pay teachers more. One mechanism is a ballot referendum for higher taxes to pay teachers. Oh, that bad word—taxes.

The results of these studies raise many questions, and surely one is the reasonableness of expecting significant student improvement after just one year. Even if an effective teacher instantly adopts an improved practice, is a year long enough to effect student outcomes? Maybe, as excellent tutors have experienced. When a teacher fills a student's gap in “profound understanding of fundamental mathematics (Liping Ma),” the improvement in math scores can be dramatic.

In fact, teachers generally do not wait for annual bubble tests to decide if a new tool is worth keeping. They look for immediate effects based on criteria residing in their own head and nowhere else. If efficacy is not quickly apparent, teachers will abandon a new practice as quickly as they took it up. A school year is short, not long. Teachers do not have time to spin their wheels.

Once a student decides school is “doing time” or a particular subject is beyond them, once a student shuts down, it is very difficult, even for the best teachers, to reawaken wonder, interest, curiosity and the ability to persevere through a challenge.

All the foregoing being said, any analysis of the effectiveness on professional development suffers from the scarcity of rigorous studies of professional development efficacy.

The most recent review of studies of the impact of teacher PD on student achievement revealed a total of nine studies that have rigorous designs—randomized control trials (RCTs) or certain quasi-experimental designs (QEDs)—that allow causal inferences to be made (Yoon et al. 2007).*

Even fewer studies have examined the ways teachers incorporate and evaluate what they learn from professional development.

What surprised me most was the observation of teacher instructional practices:

To measure instructional practice for treatment and control teachers, one classroom observation was conducted for each teacher after the treatment teachers in that district had had at least 5 of the 8 scheduled days of institutes and seminars. The observations produced three primary measures of instructional practice, which documented the frequency with which the teacher employed several key behaviors encouraged by the PD program.13 The first measure, Teacher elicits student thinking, encompassed such behaviors as asking other students whether they agree or disagree with
a particular student’s response and also included behaviors elicited from the students such as offering additional justifications or strategies. The second measure, Teacher uses representations, counted the number of times the teacher displayed and explained a visual representation of mathematics, such as number lines or ratio tables, as well as the number of different types of representations the teacher used. The third measure, Teacher focuses on mathematical reasoning, counted the number of times that the teacher asked questions such as Why does this procedure work? Why does my answer make sense? or Why isn’t 3/4 a reasonable answer to this problem?

On average, teachers elicit student thinking about 3 times per class period, use representations twice and focus on math reasoning once. Me, I do all that all day long, so it seems to me that the problem with math education is far more fundamental than the study was designed to evaluate.

Generally speaking, middle school teachers are classified as secondary teachers and are expected to have a bachelors degree in the subject area they teach. Because of teacher supply and demographic issues, teachers are often assigned subjects outside their field. Even so, the percentage of studied teachers with a math or math related degree barely crested double digits. On a baseline test of the math covered in the professional development, the studied teachers “had a 55% chance of getting the right answer,” compared to 93% chance for the professional development facilitators. The average seventh- grade student of these teachers scored in the 19th percentile on a test of fractions.

Middle school math is a classic illustration of the old canard, the blind leading the blind. Math instruction at the elementary level, where fractions are introduced, is even worse. On the other hand, “more qualified teachers had a tendency to gravitate to schools that served students from more privileged backgrounds.” **

I am not saying that ALL PD is worthless and I have written about effective PD. Nor am I saying that these two studies are the be all and end all.

I am pointing out flaws in the study and flaws in the conclusions. I am saying that perhaps the designers of the study misunderstand how teachers adopt and/or reject what they learn from PD. Perhaps the designers did not adequately account for a host of other variables.

I am saying that what passes for improvement in eliciting student thinking is not much, just one more question in the class period.

I am also implying that student test scores do not necessarily measure of success of PD.



*Yoon, K.S., Duncan, T., Lee, S. W.-Y., Scarloss, B., and Shapley, K. Reviewing the Evidence on How Teacher Professional Development Affects Student Achievement (Issues & Answers Report, No. 033). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest, 2007.

**J.E. Rockoff, "The Impact of Individual Teachers on Student Achievement: Evidence from Panel Data," American Economic Review, Papers and Proceedings, May 2004.

Quality Professional Development on the Cheap

People who have a lot of money do not have to think too hard about problems.  They can just throw a few dollars at their problems and solve them fairly easily.  They may complain the whole time they are pitching the dollars, but they pay just the same.  People without money have to be a lot more resourceful, creative and innovative.  Maybe I can't afford $300 for a diagnostic on my car when it flunks the smog certification.  So I ask around, pay $8 for a fuel additive, drive my car over the grade, and voila, it passes the certification with flying colors, and I keep most of the $300.

The very act of idea-generation can highlight areas of waste and profligate spending. A good example is professional development.

"Any district hiring a consultant to come in for a one day for $10,000 or $15,000—that's a waste of time and money," says Ed Wilgus, a former district professional development manager who is co-founder of Systemic Human Resource Solutions.

There are lots of options for planning a satisfying DIY professional development program.

Here are some ideas:
1. Veteran teachers share their best practices.
2. A conference format with several options within any particular time frame
3. A longitudinal lesson study Japanese style.
4. A collaborative discussion addressing a specific concern within the district.
5. Teachers design, conduct and share the results of active research within district classrooms.
6. Reflective analysis of the rational behind specific teaching strategies.
7. Reviews of books and journal articles of interest.
8. Demonstrations of math manipulatives and other resources.
9. Presentations by parents.
10. Classroom swapping-teaching for a day in a very different grade or subject area with no more preparation than an average substitute teacher.

One school district had a great idea for using the Internet.

Instead of hiring presenters to come to their schools, they downloaded free archived video presentations from the Web site of the K-12 Online Conference, an annual grassroots gathering of instructional technology aficionados. Then they featured the videos as part of a special series of staff development sessions on technology topics. Members of the district's tech team and Web-savvy teacher leaders facilitated the sessions, leading discussions on the presentations and addressing teachers' practical concerns. In some cases, they even conducted live follow-up interviews with the original presenters via Skype, the free Internet phone service.

P.S. A school can get me for $1000-$1500, (not $10000-$15000), depending on the amount of time, distance, hotel, etc. I do presentations on math, science and foreign language teaching, ESL, art, curriculum and the Japanese education system. I help teachers plan and organize their own self-facilitated professional development. I also write education grants for schools.

Teachers Misbehaving

Purveyors of professional development don't like it much when the teachers they are supposed to be teaching misbehave. I have been on both sides of the podium. Teachers have good reason for regarding professional development sessions as a monumental waste of time. Usually the sessions are mandatory, and the presenter has been commissioned to present specific information whether or not the audience is interested. One school district asked me to present a hands-on literacy session to high school teachers on developing questions that build critical thinking.

It quickly became clear that what these high school teachers really wanted was tips on phonics, and how to integrate phonics into their content lessons. I converted my presentation to phonics on the fly. Because I had not brought any phonics materials with me, I had to abandon the hands-on part. I ended up delivering mostly lecture, but the teachers loved it. Six months later I was commissioned to present an overview of the six traits of good writing to a group of elementary teachers in a completely different geographical area. What the teachers wanted to do was vent their complaints about the district's adoption of the reading curriculum, Success for All. When they were done, I carried on with my presentation. Only later I found out that the administration canceled my future presentations because I did not shut down the criticism.

So you never know.

For every expectation, there are other conflicting expectations. If some teachers expect presenters to model hands-on techniques, other teachers want the presenter to just tell them the info and let them go back to grading papers. The presenter comes prepared with the information commissioned by the administration only to find the audience deems the information irrelevant to their needs. I gained appreciation and sympathy for presenters when I became a presenter.

The worst problems with professional development can be avoided by letting teachers create their own professional development. Administrators are notoriously out of touch with the teaching staff.

This Professional Development Model Works

In one room, one of our own high school math teachers was helping colleagues learn to use The Algebra Lab. Down the hall, another district teacher was showing a group of colleagues how to set up a marine aquarium in their classrooms. On the second floor a second-grade teacher showed colleagues how she implemented Grace Pylon's Workshop Way in her classroom. Elsewhere a parent shared poignant stories of the struggle and victories her family experienced every day with her autistic son. In the library yet another district teacher shared slides of her summer research with penguins. There was even a bona-fide outside presenter, one of the original whole language researchers.*

Oh, those teacher work days. Students love the day off from school. Parents begrudge the additional child care burden. What do students and parents imagine teachers do on teacher work days? Because work days are so often scheduled near the end of a term, they might imagine teachers are grading papers and calculating grades for report cards. Teachers wish, but more likely they are captive to yet another worthless professional development workshop commissioned by the district administration. Teachers resent most of the professional development they are forced to endure. But teachers loved professional development in my district.

Education Week recently sponsored a discussion of professional development. The moderator introduced the topic of discussion:

Teachers are often dissatisfied (to put it mildly) with school/district professional development offerings. If an administrator or policymaker asked you how professional development for teachers could be improved, what would be your advice? What do schools commonly do wrong in providing professional development? Alternatively, what sort of PD experiences have you had that really worked and benefited your instruction and that you would like to see more of?

What teachers want in professional development is the ability to choose in-service workshops based on input from their self-evaluation of needs and interests. They want workshops presented by qualified presenters. These presenters do not have to be affiliated with a professional development company or university.

What they do not want:
1. Required attendance to the only possible choice.
2. Commissioned summarizers of other researchers' studies.
3. Presenters who waste time with obligatory hands-on activities.
4. Fly-by-night presenters unavailable for follow-up.
5. Arbitrarily pre-filtered presenters.

The last bullet may require further explanation. Districts are fond of purchasing professional development from a professional development company or a university. These presenters fly in, present, and fly out. Worthy individuals with useful messages cannot get heard because they do not work for the contract company or university, and the district has either committed all the available professional development budget or signed an exclusive contract with a company or university. Teachers would appreciate other alternatives. In the example of one comment to the Education Week discussion:

I also attend non-ceu workshops on my own time and would like to see some way to have these count at PD. For example, just because a bona fide rocket scientist at Pratt and Whitney is not a state recognized ceu provider, his seminar on calculators in the classroom was just plain excellent, and far beyond anything a vendor or such would offer. Here was someone who manages professional scientists, uses calculators daily, is cognizant of all the devices and software that are available, consults with  schools, yet he is not a 'recognized' ceu provider. Just plain silly, to me.

Professional development designed in-house is far less expensive and far more effective than the pricey canned stuff generally foisted on teachers.

Here are some ideas:
1. Veteran teachers share their best practices.
2. A conference format with several options within any particular time frame
3. A longitudinal lesson study Japanese style.
4. A collaborative discussion addressing a specific concern within the district.
5. Teachers design, conduct and share the results of active research within district classrooms.
6. Reflective analysis of the rational behind specific teaching strategies.
7. Reviews of books and journal articles of interest.
8. Demonstrations of math manipulatives and other resources.
9. Presentations by parents.
10. Classroom swapping-teaching for a day in a very different grade or subject area with no more preparation than an average substitute teacher.

When I taught for the Department of Defense Dependent Schools, we teachers designed and implemented our own annual Educator's Day, held on a Thursday and Friday in March, complete with vendors and community booths. The entire district took over the high school for these two days. Each fall, at least one teacher from every school volunteered to be part of the planning committee. We solicited proposals from presenters among our colleagues, parents and the general community. We sent out surveys asking teachers what they wanted to hear. We also collected evaluation forms on every presentation that included a request for suggestions for the next year. We usually extended a special invitation to a noted researcher to present their original research. These original researchers were a highlight. Teachers appreciated hearing research “from the horse's mouth," so to speak.

No one took attendance at our educator days, so presumably if a teacher stayed home, or took a long weekend vacation, no one would know. But Educator's Day was so valuable, no one stayed away. We even included opportunities for social time when teachers could interact privately with the presenters and colleagues from other schools. Even the general public attended workshops. Each year saw greater success than the year before.

If you need help designing your own Educator's Day, you may contact me at the email address of the blog, or by writing a comment to this entry.

You may also contact the Fund for Teachers. Fund for Teachers solicits proposals from teachers for summer travel sabbaticals. Individual grants are worth $5000; team grants are worth $10,000.


*Footnote. Much later, I discovered California had failed in its implementation of “whole language,” however California's version turned out to be something far different than what the original researchers had presented to us.