Of the factors districts use to select textbooks, student achievement impacts are almost never considered, because rarely do such data exist. A new study by Rachana Bhatt and Cory Koedel aggregated this data to look at the elementary mathematics textbooks chosen by Indiana districts for the 1998-2004 cycle and shows one program to be clearly more effective than two others.
The kicker: that series--Silver-Burdett Ginn--is no longer available.
Another kicker: the least effective series identified in their evaluation - Saxon Mathematics - did not lose market share significantly between 1998 and 2004.
(The third series was Scott Foresman-Addison Welsey. Together these three accounted for 86 percent of the textbooks used by districts during this period.)
The analysis was possible through the combination of a data set from Indiana (one of only two states -- the other is Florida -- that maintains current and historical district textbook adoption data) with school and district demographic and test data.
As far as the authors have been able to determine, theirs is the largest curriculum evaluation to date, covering 716 schools in 213 districts. They estimate that the average effect of adopting SBG versus Saxon was equivalent to .13 standard deviations (on the student tests used as the outcome measure). This estimate is in line with the curriculum work of Grover Whitehurst and Matt Chingos, which we highlighted back in April.
Bhatt and Koedel decry the lack of data about textbooks, noting that it would be straightforward and inexpensive for states to systematically capture it. They will be publishing an online technical appendix to serve as a resource for researchers wishing to conduct similar evaluations in the future.
We hope many take advantage of these resources, because evaluations like this open up a large number of potential research questions. For example: SBG and Saxon are viewed as traditional mathematics curricula, yet one clearly outperformed the other, suggesting that the traditional versus reform math curriculum debates may be drawn along incorrect lines.
--Rob Rickenbrode
The kicker: that series--Silver-Burdett Ginn--is no longer available.
Another kicker: the least effective series identified in their evaluation - Saxon Mathematics - did not lose market share significantly between 1998 and 2004.
(The third series was Scott Foresman-Addison Welsey. Together these three accounted for 86 percent of the textbooks used by districts during this period.)
The analysis was possible through the combination of a data set from Indiana (one of only two states -- the other is Florida -- that maintains current and historical district textbook adoption data) with school and district demographic and test data.
As far as the authors have been able to determine, theirs is the largest curriculum evaluation to date, covering 716 schools in 213 districts. They estimate that the average effect of adopting SBG versus Saxon was equivalent to .13 standard deviations (on the student tests used as the outcome measure). This estimate is in line with the curriculum work of Grover Whitehurst and Matt Chingos, which we highlighted back in April.
Bhatt and Koedel decry the lack of data about textbooks, noting that it would be straightforward and inexpensive for states to systematically capture it. They will be publishing an online technical appendix to serve as a resource for researchers wishing to conduct similar evaluations in the future.
We hope many take advantage of these resources, because evaluations like this open up a large number of potential research questions. For example: SBG and Saxon are viewed as traditional mathematics curricula, yet one clearly outperformed the other, suggesting that the traditional versus reform math curriculum debates may be drawn along incorrect lines.
--Rob Rickenbrode