Education technology companies are increasingly promoting artificial intelligence as a way to save teachers鈥� time鈥攅specially when it comes to lesson planning.
But when it comes to crafting engaging lessons that make good use of technology and activate students鈥� critical thinking skills, AI gets a failing grade, according to researchers who analyzed more than 300 civics lesson plans generated by three popular large language models.
Overall, the lessons generated by AI mostly engaged lower-order thinking skills鈥攍ike memorization鈥攁s opposed to deeper ones, like analysis or creation, said the researchers, who were scheduled to present their findings at the ISTELive 25 + ASCD Annual Conference 25 in San Antonio, June 29 to July 2.
The teacher has to formulate their own ideas, their own plans. [Then they could] turn to AI, and get some additional ideas, refine [them]. Instead of having AI do the work for you, AI does the work with you.
The AI-generated lessons largely failed to include multicultural, diverse, or inclusive content. And they didn鈥檛 creatively incorporate interactive education technology, instead directing teachers to show a video or create a PowerPoint presentation.
If a teacher were to take one of these lesson plans and implement it as written, their students would be 鈥渏ust getting a conventional textbook represented in a different way,鈥� said Robert Maloy, a senior lecturer at the University of Massachusetts-Amherst and a former social studies teacher.
That鈥檚 not a strong foundation for civics, Maloy said. 鈥淎s a member of a community who is civically engaged, you are taking action. You are formulating viewpoints. But in these [AI-created] learning plans, you are not ever getting very many opportunities to do that.鈥�
AI use in classrooms is already widespread
Those conclusions are important given that nearly two in five teachers鈥�38%鈥�are already using AI to plan lessons, according to a nationally representative survey of 1,135 糖心动漫vlog鈥攊ncluding 731 teachers鈥攃onducted between Sept. 26 and Oct. 8 of last year by the EdWeek Research Center.
Maloy and his co-author, Torrey Trust, a professor of learning technology at UMass-Amherst, asked three Generative AI chatbots鈥擮pen AI鈥檚 ChatGPT, Google鈥檚 Gemini, and Microsoft Copilot鈥攖o develop lesson plans for each of the 53 standards within the Massachusetts 8th Grade History & Social Science Curriculum Framework.
They used two different prompts for each standard. One prompt asked the chatbots to create a lesson for a particular standard, without describing what it should look like. For example, the researchers asked the bots to 鈥渨rite a lesson plan for the 鈥�
The other prompt was similar, but just a bit more specific. They asked the large language models to come up with a 鈥渉ighly interactive鈥� lesson plan for a particular standard. That meant the prompt changed to 鈥渨rite a highly interactive lesson plan for the Articles of Confederation and Shay鈥檚 rebellion.鈥�
AI-generated lessons not much better than reading a textbook, researchers find
Researchers analyzed the resulting 310 lesson plans and 2,230 activities through three different lenses: Did the lesson engage students鈥� higher-order thinking skills like analysis and creation, as opposed to simply memorization? Did it integrate diverse, inclusive, or multicultural content? And did it make smart use of the technology?
The lessons were unimpressive in all three areas.
For instance, when it comes to higher-order thinking skills鈥攁s defined by Blooms Taxonomy, a widely used education framework鈥攋ust 2% asked students to evaluate. Only 4% asked for analysis and/or creation. By contrast, about half鈥�45%鈥攃alled for students to 鈥渞emember鈥� information presented, which is considered a lower-order thinking skill.
There鈥檚 limited diversity and inclusion
And only 25% of lessons crafted by ChatGPT included some sort of activity that called for students to consider the contributions of individuals from marginalized groups or delve into the issues impacting such communities.
For instance, one lesson asked students to explore how the concept of judicial review worked in Brown v. the Board of Education, the Supreme Court ruling that led to the end of segregated schools, or in Obergefell v. Hodges, which legalized same-sex marriage.
The results for Gemini and Copilot were similar. Twenty-three percent of Copilot鈥檚 lessons included some sort of multicultural twist, while 27% of Gemini鈥檚 did.
What鈥檚 more, those percentages may actually be artificially high, Trust said.
The activities featuring diverse, inclusive, or multicultural content showed up almost exclusively in lesson plans for standards that focused on diverse groups. While Massachusetts鈥� social studies standards prioritize such content, that鈥檚 not the case in many states, Trust said.
Tech tools that don鈥檛 use tech well
Despite being built on cutting-edge platforms, the lesson plans made minimal use of actual educational technology. That includes 11% of those generated by ChatGPT, 1% of those generated by Gemini, and 16% of those generated by Copilot.
鈥淔or these cutting-edge technologies, their integration of technology in lesson plans was quite abysmal,鈥� Trust said.
Interestingly, the specificity of the prompt鈥攚hether general or requesting a 鈥榟ighly interactive鈥� lesson鈥攈ad little effect on the quality of the resulting plans.
Don鈥檛 rely on AI to teach, but use it to brainstorm
Despite the somewhat dismal results, teachers can still use AI to inform lesson development, Trust said. They just shouldn鈥檛 expect that chatbots are going to come up with a great lesson all on their own.
Instead, the chatbots might be helpful in brainstorming, Trust and Maloy said. For instance, teachers could prompt the platforms to come up with 10 different ways for teaching Shays鈥� Rebellion. Or they could ask for feedback on a lesson by writing a prompt along the lines of 鈥渉ere鈥檚 my plan for teaching Shays鈥� Rebellion, give me 15 ways to improve it.鈥�
They could also be more specific, asking AI to generate lesson plans that specifically speak to the highest levels of Bloom鈥檚 taxonomy.
鈥淭he teacher has to formulate their own ideas, their own plans,鈥� Maloy said. Then they could 鈥渢urn to AI, and get some additional ideas, refine [them]. Instead of having AI do the work for you, AI does the work with you.鈥�
