While artificial intelligence brings many new challenges into the classroom, it may also offer opportunities to ease the work of teachers and administrators. New research asks whether AI can provide timely and targeted feedback on teachers' instruction. And if it's possible, will teachers want and use that feedback?
New research suggests that this feedback is indeed possible and can improve teachers' instruction—but teachers still have their doubts.
Past research found that an AI program could give teachers feedback on their "uptake of student ideas" when they provide online instruction from a scripted curriculum. This development, while promising, was not relevant for many teachers, who tend to teach while moving around a classroom rather than sitting at a computer.
A new working paper by the same lead author (Dorottya Demszky) and her colleagues finds that automated feedback can also work in a "brick and mortar" classroom setting. This study uses an existing app (TeachFX) through which teachers record full lessons; the app then creates a transcript of their lesson and provides feedback on specific elements.
The study layered on an additional element of feedback to what the app already offers. The AI program provided a sample of several hundred Utah teachers with insight into their use of "focusing questions," which "guide the student based on what the student is thinking rather than how the teacher would solve the problem." Examples include, "What decisions did you make when you were placing 3/5 on the number line?" and "What do you already know about slope?" The researchers used math instructional coaches to train a language model to identify examples of focusing questions within the lesson transcript. This fine-tuned model had 84% accuracy in identifying focusing questions.
Teachers were randomly assigned to either use the standard TeachFX app or to be in the treatment group that received the additional focusing questions feedback via the app's platform and regular emails.
Importantly, treatment teachers did increase their use of focusing questions, with about a 20% increase compared with control group teachers. This positive outcome is notable since teachers had limited engagement with the app: Only a little over half of teachers regularly opened emails with the insights (a third of teachers never opened an email), and less than half ever went to the platform's more detailed insights page. However, treatment teachers did visit the platform more often than control teachers, perhaps indicating that the emails were a helpful reminder about the content included on the site.
Interviews with the study participants (both treatment and control teachers) revealed a range of concerns, including worries about the quality of both the transcript and feedback. Yet even with these concerns and limited use of the app, this targeted feedback approach clearly influenced teachers' instruction.
While this kind of feedback should not take the place of full in-person observations by real people, it offers a great deal of promise to help teachers (and perhaps aspiring teachers) with timely and targeted feedback on specific skills.
New research suggests that this feedback is indeed possible and can improve teachers' instruction—but teachers still have their doubts.
Past research found that an AI program could give teachers feedback on their "uptake of student ideas" when they provide online instruction from a scripted curriculum. This development, while promising, was not relevant for many teachers, who tend to teach while moving around a classroom rather than sitting at a computer.
A new working paper by the same lead author (Dorottya Demszky) and her colleagues finds that automated feedback can also work in a "brick and mortar" classroom setting. This study uses an existing app (TeachFX) through which teachers record full lessons; the app then creates a transcript of their lesson and provides feedback on specific elements.
The study layered on an additional element of feedback to what the app already offers. The AI program provided a sample of several hundred Utah teachers with insight into their use of "focusing questions," which "guide the student based on what the student is thinking rather than how the teacher would solve the problem." Examples include, "What decisions did you make when you were placing 3/5 on the number line?" and "What do you already know about slope?" The researchers used math instructional coaches to train a language model to identify examples of focusing questions within the lesson transcript. This fine-tuned model had 84% accuracy in identifying focusing questions.
Teachers were randomly assigned to either use the standard TeachFX app or to be in the treatment group that received the additional focusing questions feedback via the app's platform and regular emails.
Importantly, treatment teachers did increase their use of focusing questions, with about a 20% increase compared with control group teachers. This positive outcome is notable since teachers had limited engagement with the app: Only a little over half of teachers regularly opened emails with the insights (a third of teachers never opened an email), and less than half ever went to the platform's more detailed insights page. However, treatment teachers did visit the platform more often than control teachers, perhaps indicating that the emails were a helpful reminder about the content included on the site.
Interviews with the study participants (both treatment and control teachers) revealed a range of concerns, including worries about the quality of both the transcript and feedback. Yet even with these concerns and limited use of the app, this targeted feedback approach clearly influenced teachers' instruction.
While this kind of feedback should not take the place of full in-person observations by real people, it offers a great deal of promise to help teachers (and perhaps aspiring teachers) with timely and targeted feedback on specific skills.