From the Cool Cat Teacher Blog by Vicki Davis
Subscribe to the 10 Minute Teacher Podcast and Cool Cat Teacher Talk anywhere you listen to podcasts.
Do we have any PhD candidates out there? I would argue that at this point in history, your research might matter more than ever. Here’s an example that shows why.
Patrick O’Neil at a community college in Indianapolis has literally set AI in education on its head. So, he read the research on AI, and something didn’t add up. Justin Reich, associate professor of digital media at MIT, shared this on LinkedIn, said:
So he [Patrick] did what any normal person would do: 1) he read 40 papers and chapters on meta-analytic methods teaching himself the field, 2) found and carefully read 14 meta-analyses on GenAI and education, 3) read a sample of 60 of the underlying papers in these analyses. Finally, he did 4) published a forensic review of these 14 papers.
If you love educational research as I do—if you love helping practicing teachers get evidence to support good practice—I suspect you will find the review very disturbing.
None of the 14 papers have a coherent outcome measure, lumping indicators of mental health alongside tests alongside self-reports of perceived learning. 11 of 14 have no study population criteria, so children can be included alongside medical trainees. Six of the meta-analyses include a paper with fabricated citations. Three meta-analyses include a study with references published before November 2022 that refer to ChatGPT. The main paper summarizes and illustrates these problems, and the included appendices provide a review of each meta-analysis. I find Patrick’s work to be thorough and compelling.
The operative word there. Patrick read. He read. He thought. He examined. And then, Patrick wrote. A chemistry teacher at a local university didn’t like what he read. Look at this!
As I was reading Patrick’s paper, I remembered an interview I did with Dr. Maggie Broderick about servant scholarship and the need for research at the graduate level that makes education a better place.
Honestly, from what I saw Patrick do, it seems like real research (where you set aside the AI and actually read the papers yourself and follow every link) may add more value than anyone can imagine. In the age of AI, Patrick proves that it is humans who actually do the work who excel. And that is the kind of servant scholarship we all need.
So, if you’re choosing your dissertation topic and you’re in the field of education, I hope you’ll consider research that can be part of your service to humankind. If you’re going to use your brainpower, use it for good. Serve with your study! Humans are more important than ever! Maggie really is encouraging for PhD students. Let me hear from you. What do you think?
“It’s much more of a process than a product.”
Dr. Maggie Broderick
Associate Professor, National University
Listen to the Show