Pity the new district superintendent. Like any responsible educational leader, he'd like to be sure that his district's curricular materials and interventions are grounded in solid scientific research. But no sooner does he start talking with his staff, his teachers, and various and sundry "experts" than he finds that everything is "research-based," including approaches that are clearly very different from those employed by his teachers. Should he let well enough alone, or should he introduce programs that seemed to work fine in the last district he was in?

Neither. Instead, he should go read Dan Willingham's ingenious new book, When Can You Trust the Experts? The book won't tell him which programs to use, but it will help him think through -- and, in some cases, see through -- the claims their creators make on their behalf. An accomplished cognitive psychologist at the University of Virginia and the author of the must-read Why Don't Students Like School? (as well as an NCTQ advisory board member), Willingham aims to make district superintendents, principals, teachers and parents into educated consumers of education research.

Willingham shows that it won't be enough for the superintendent to ignore research that might have been paid for by the people who made the curricula, programs and products he's reviewing. Nor can he simply ignore all research that isn't peer-reviewed. Like anyone else, the new superintendent's thinking will be subject to the kind of systemic biases -- "cognitive dissonance," "risk aversion" and so forth -- which psychologists have studied for years, and which Willingham ably explains in this book.

Fortunately, when applied with sufficient discipline, the scientific method can serve as a potent corrective to flawed thinking. And even non-scientists can use it to sort through the innumerable programs that we are told the "research" supposedly demands we implement. With admirably clear prose and plenty of memorable examples, Willingham walks the reader through an analytical procedure that she can use to test the claims made on behalf of any educational intervention. But Willingham is also very clear about the limits of his procedure and scientific research more generally. Cognitive scientists can no more prescribe to teachers the one best way to teach than physicists can tell architects the one best way to design a house.

Still, as Willingham notes, it remains striking how frequently the field of education seems to get swept up by fads that on closer inspection turn out to be revivified approaches knocked down many years before ("whole language," anyone?). To combat these zombie theories, Willingham holds out the hope that teachers unions will take on the same role as professional medical associations, which sift through the enormous and burgeoning clinical research literature and, where appropriate, make changes to established "standards of practice."

But don't schools of education also have a role to play here? Ed schools based in research universities might do well to follow the example of the University of Michigan and systematically investigate and, where appropriate, disseminate "high-leverage teaching practices" to support learning. And all preparation programs should make sure their candidates understand the current state of research relevant to classroom practice and equip them to sort the wheat from the chaff. Assigning them When Can You Trust the Experts? would be an excellent place to start.