When the AI chatbot ChatGPT initially launched in late 2022, it quickly became the subject of much debate. Depending on who you ask, the technology is either going to revolutionize everything for the better or accelerate our very downfall. But no matter where you stand, you’ve probably used it once or twice—maybe even on a daily basis, asking it to proofread your emails. That convenience, however, might come at a risk. According to a new study led by MIT researcher Nataliya Kos’myna, using ChatGPT appears to have a negative impact on how we learn.
In a lengthy conversation with astrophysicist Neil deGrasse Tyson on his StarTalk podcast, Kos’myna discusses her recently published paper, “Your Brain on ChatGPT,” which focuses on an experiment involving 50 students from the greater Boston area. They were divided into three groups—one using ChatGPT, one using Google, and one using only their brains—and asked to quickly write essays on “high-level” topics like “What is happiness?”, “Is there a perfect society?”, and “Should you think before you talk?” EEG headsets monitored their “brain functional connectivity,” and follow-up questions documented their thoughts on the process. Later, the students swapped places with other groups.
“Brain functional connectivity”
This form of brain activity essentially looks at “who talks to who in the brain.” Kos’myna gave the example of deGrasse Tyson doing an interview without notes: “Really, your brain [is] on fire, so to [speak],” she said. “‘OK, what was her name again? Where was the study? What is happening?’ You need to really push through with your brain.” The study ultimately found that connectivity is “significantly higher” for the brain-only group, compared to the two others, with the ChatGPT group showing the least.
“The brain doesn’t really struggle when you use this tool, so you have much less of this functional connectivity,” she said. “But what is, I think, interesting [is that]…first of all, what we found is that the essays were very homogeneous. The vocabulary that was used was very, very similar for the [ChatGPT] group. It was not the case for the search-engine and for the brain-only group.”
Equally interesting were the students’ follow-up responses, given 60 seconds after submitting their essays. They were asked to provide a quote of any length from their writing, and 83% of participants could not offer a single line—marking a discrepancy with the other two groups. In addition, 15% of the ChatGPT users said they didn’t feel any “ownership” over the work. “I think that’s where it actually gets really tricky because if you do not feel that it’s yours but you just worked on it, does this mean that you do not care?” Kos’myna asked. “We didn’t obviously push it that far in the paper, but I think this is something that definitely might require much further investigation. If you don’t care, you don’t remember the output, you don’t care about the output, what ultimately is it for? Why were you even here, right?”
“Cognitive load”—but hopefully not overload
It ultimately all comes back to “cognitive load.” “The whole idea,” the researcher says, “is that it’s how much of the effort you would need to be on the task or to process information in the current task.” And that load, Kos’myna argues, is essential to the learning process. We need our brains to work hard, to be pushed to some degree, while also avoiding cognitive overload. “Information [that is] already delivered to you within 30 seconds or 3 seconds or 10 seconds, and you haven’t really struggled, there is not a lot of this cognitive load,” she says. “A lot of people would say, ‘Oh, that’s awesome. That’s kind of the promise of a lot of these LLMs [Large Language Models] and a lot of these tools.’ But we do not want to make it too simple, right? We do not want to take away this cognitive load…I know it sounds like, ‘Cognitive load? Don’t we want to take it away?’ No, we actually do not want to take it away.”
It’s worth spending some time exploring the actual study. In the conclusion, the authors write, “The LLM undeniably reduced the friction involved in answering participants’ questions compared to the Search Engine. However, this convenience came at a cognitive cost, diminishing users’ inclination to critically evaluate the LLM’s output or ‘opinions’ (probabilistic answers based on the training datasets). This highlights a concerning evolution of the ‘echo chamber’ effect: rather than disappearing, it has adapted to shape user exposure through algorithmically curated content.”
As numerous articles have noted in recent years, ChatGPT is likely here to stay. The question, clearly, is how we make the best use of it while keeping our cognitive load at the right level.
