It鈥檚 a common sentiment from English teachers facing an onslaught of AI-generated writing in their classrooms: When students rely on ChatGPT to write their essays, they aren鈥檛 engaging their brains as deeply about the topic as they would if they did all the writing themselves.
Now, it鈥檚 also a finding backed by neuroscience research.
A new from researchers at the Massachusetts Institute of Technology, Wellesley College, and the Massachusetts College of Art and Design found that giving writers free reign to use AI as much as they wanted led to some bleak outcomes.
Participants, mostly undergraduate and graduate students, who constructed essays with the assistance of ChatGPT exhibited less brain activity during the task than those participants who were asked to write on their own. The AI-users were much less likely to be able to recall what they had written and felt less ownership over their work. Independent evaluators who reviewed the essays found the AI-supported ones to be lacking in individuality and creativity.
But if participants wrote essays on their own first, and then used AI to write on the same topics, the results changed. This group of writers showed an increase in brain activity.
鈥淲hat it could potentially tell us is that timing could be very important for when you integrate these tools,鈥� said Nataliya Kosmyna, a research scientist at the MIT Media Lab and the lead author of the paper.
If writers spend time thinking about their topic and collecting their thoughts before turning to generative AI, it鈥檚 possible that they could benefit more from using the tool, she said. 鈥淢aybe now you can ask questions, go back and forth. You have your opinions on the topic, you can prompt in different directions.鈥�
The paper is a preprint, meaning it hasn鈥檛 yet been peer reviewed and published in an academic journal. And the researchers only worked with a small sample of participants, all of whom were undergraduate students, graduate students, or university employees.
Still, the findings could offer important clues about when generative AI use might short-circuit the learning process鈥攁nd when it might actually deepen students鈥� thinking.
鈥淚t鈥檚 the kind of study that we need to get a handle on what鈥檚 happening when you use ChatGPT,鈥� said Steve Graham, a professor who studies writing instruction at Arizona State University.
Study found less brain activity in writers using ChatGPT鈥攅xcept in one scenario
In the study, researchers asked 54 participants to write three essays each, across three separate sessions, responding to prompts drawn from the SAT college-entrance exam. They sorted the participants into different groups.
In the first group, participants could use Open AI鈥檚 GPT-4o however they wanted to assist in the writing process. In the second, participants could search the internet and use any website to inform their writing鈥攅xcept ChatGPT or any other large-language AI model. Participants in the third group couldn鈥檛 use any research tools; the researchers called this group 鈥渂rain-only.鈥�
The researchers monitored participants鈥� brain connectivity during the tasks through electroencephalogram, or EEG, a tool that measures electrical activity in the brain. They also asked participants a series of questions after the tasks and had independent raters鈥攖wo English teachers and one AI鈥攕core their essays.
EEG results demonstrated that the group of writers who didn鈥檛 rely on any outside support had the strongest, widest-ranging neural activity. The group that used ChatGPT had the least activity, and the group that searched the web fell somewhere in between.
The brains of the writers who were working without research aids lit up in areas related to coming up with creative ideas, integrating multiple pieces of information, and self-monitoring鈥攁 pattern that 鈥渦nderscored the high internal demand for content generation, planning, and revision in the absence of external aids,鈥� the researchers wrote. Those features of organizing thoughts, outlining, and revising, are core to the writing process.
Participants in the ChatGPT group also had a harder time remembering what they wrote. When asked to quote from their own essay, 83% of the group couldn鈥檛 do so. Only 11% of the participants in the search engine group and the brain-only group failed at the same task.
Is it your own essay if AI helped you write it?
And while almost all the brain-only group said they felt that their essays were their own, the ChatGPT group wasn鈥檛 as sure. This group 鈥減resented a fragmented and conflicted sense of authorship,鈥� the researchers wrote. 鈥淪ome participants claimed full ownership, others explicitly denied it, and many assigned partial credit to themselves.鈥�
In evaluations of the essays themselves, the two English teacher judges wrote that AI-crafted writing stood out, due to 鈥渁 close to perfect use of language and structure while simultaneously failing to give personal insights or clear statements.鈥� The AI judge, by comparison, couldn鈥檛 tell the difference between essays written with and without AI鈥檚 help.
After participants had written the three essays, the researchers asked everyone back for a fourth session鈥攚ith one twist. Those assigned to the brain-only group had access to ChatGPT, while those who initially could have used ChatGPT were asked to write unaided. Eighteen of the original participants completed this fourth session, in which they picked one of the three essay topics they had already written on, so they were familiar with the subject matter.
When the ChatGPT group switched to writing without support, they didn鈥檛 reach the same level of brain activity as the group that never had access to AI.
But when the brain-only group switched to using ChatGPT, they saw an increase in brain connectivity.
鈥淔rom an educational standpoint, these results suggest that strategic timing of AI tool introduction following initial self-driven effort may enhance engagement and neural integration,鈥� the researchers wrote.
Students need time to develop writing skills and form ideas without AI, 糖心动漫vlog say
For Brett Vogelsinger, a high school English teacher in Bucks County, Pa., the findings offer a warning: 鈥淥utsourcing too early is the biggest risk here,鈥� he said.
Vogelsinger is the author of Artful AI in Writing Instruction and has incorporated the tool into some of his lessons. Still, he said, 鈥測ou need to have some writing time that鈥檚 you, and the paper, and your thoughts.鈥�
This could mean incorporating AI only for revision, said Kristina Peterson, an English teacher at Exeter High School in Exeter, N.H., and the co-author of AI in the Writing Workshop with her colleague, English teacher Dennis Magliozzi.
If students are using AI as a writing partner, Magliozzi said, 鈥測ou have to show up to the table having written first.鈥�
Beyond giving students the space to develop original ideas, the study also underscores the importance of teachers helping students learn writing skills and an understanding of the writing process, said Graham, the ASU professor.
Even if students come up with their own ideas, they lose out on exercising important writing muscles if they ask ChatGPT to arrange those ideas into an essay for them, he said. 鈥淲e鈥檙e not going to learn a lot about sentence-construction skills,鈥� Graham said, as an example. Writing tends to use more sophisticated syntax and vocabulary than oral language鈥攕yntax that students need to practice.
Targeted writing instruction also helps students learn how to structure their work in a logical, clear way.
Lacking these skills hampers students鈥� ability to write strong essays鈥攁nd it also makes it harder for them to determine whether AI-generated content is high quality. If students are using ChatGPT as a writing aid, Graham said, 鈥渨e need to make sure to have the skills we need to determine, did it meet our intentions?鈥�
It鈥檚 not clear just how much writing expertise students do need before they can make good judgments about the quality of an AI鈥檚 output, said Kosmyna, the study鈥檚 lead author. She stressed that this study鈥檚 sample was quite small: 54 participants completed the first three sessions, and only 18 came back for the fourth.
For now, it鈥檚 important to carefully consider how teachers say AI could be best used鈥攁nd to heed their warnings, she said.
鈥淲e do need to support them, and listen to them, and not just push randomly all the tools under the sun on them without understanding.鈥�
