Welcome back to the third and final blog post in our series on how LIB100 has (and hasn’t) changed in the AI era.

As we complete our blog series on how our credit-bearing information literacy course, LIB100: Academic Research and Information Issues, has been adapted for the generative AI era, we’d like to pause for a moment of silence for the pre-AI research project…

Of course, the academic research project (especially the academic research paper) will live on, but it will likely never look the same as it did in the pre-AI era.

In our last post, we chatted about the shift from a search first environment to an answers first environment, and how that has led us to spending more time in class on evaluating claims over evaluating sources. This shift has also led us to reconsider what skills and knowledge students need in order to successfully ‘catch’ AI errors in the academic research process, including the ability to carefully read scholarly articles and recognize a work’s relationship to other pieces of scholarship. We realized we had been making assumptions about students’ abilities to read deeply, especially complex, technical texts that require sustained attention.

What is Research For?

The traditional academic research paper has always had its problems and critics, including those described in Barbara Fister’s contribution to Project Information Literacy’s “Provocation Series.” When we ask students to read and comment on this article at the beginning of LIB100, we discuss some of these criticisms, including the problematic framing of inquiry as the pursuit of a winnable argument and evidence as a “consumer choice,” as well as an emphasis on answers over inquiry in general. Instead, Fister encourages us to consider why we do research in the first place, and to grapple with how we might reframe research for a highly distrustful sociopolitical moment. Then and now, what was the point of library research? Is it a worthwhile endeavor to continue to teach research skills, when generating a complete research output is easily available in both free and licensed tools, including AI tools that our own library licenses?

We would argue YES, it is still worthwhile, and the introduction of AI has provided us with the opportunity to use principled thinking to revisit the purpose of library research (i.e., to understand and contribute to an ongoing conversation) and rebuild some of our key assignments to reflect these first principles in light of AI. Furthermore, where can deep learning still exist within the research process? What knowledge and skillsets will be valuable in an unpredictable future?

From Hallucinations to Misgrounding

If the purpose of research is to understand and contribute to an ongoing conversation, then one of the biggest risks AI poses isn’t only that it can produce a finished research product for students. It’s also that it can misrepresent the scholarly conversation itself in ways students may not yet be equipped to catch.

One of the most obvious and widely discussed problems with AI tools is their propensity to hallucinate, bullshit (Frankfurt’s definition), misattribute, or *whatever you prefer to call it*. For the past few years, we had focused on general-use AI tools fabricating scholarly article citations, which worked well with students for a while. As companies release newer versions of AI software and increasing numbers of students pay for premium versions of tools, we have noticed that this issue seems to be on a slow decline (though it is still an issue). This has shifted our focus to what some are calling “misgrounding,” which is when a real source is cited, but the generated content does not accurately represent the cited work. One team of researchers looking at 58,000 grounded, generated claims in answer to common medical questions users submitted to popular AI tools found “between 50% and 90% of LLM responses are not fully supported, and sometimes contradicted, by the sources they cite.” Researchers have also pointed out that novice learners are more susceptible to being misled by inaccurate AI outputs. In our class, if the generated citation passed the “looks good to me” test and included a link or DOI, students assumed it checked out without following the link or searching for the citation in a known index, to say nothing of their ability to carefully critique a generated, grounded summary.

One way we have illustrated misgrounding with students is by discussing how we may inadvertently influence generated summaries of verified scholarly sources, including RAG (retrieval-augmented generation)-enabled outputs. We decided to follow up on prompts asking for retrieval of a scholarly source with questions asking for opposing interpretations of the article in question. For example, person 1 may ask what a study says about why using peptides for wrinkles works, while person 2 asks what the same study says about why using peptides for wrinkles does not work. With all of the tools we tested, both general and research-specific, we noticed different conclusions based on how the tools are prompted. When using leading questions, ChatGPT drew markedly different conclusions from the same article. We noticed less extreme differences in Consensus, an AI tool marketed specifically for scholarly research, but different prompt phrasing drove different outputs in both summarizing findings and source rankings. While this exercise emphasizes the consequential nature of prompt phrasing, it also illustrates the need to double check claims against the original content.

That said, double-checking a claim against its source assumes students can deeply read and understand that original source, and that was an assumption we had to revisit.

Back to Basics – Breaking Down the Research Process

After the onset of modern, popular AI tools, we began to feel the research assignments we had traditionally assigned in LIB100, including annotated bibliographies and synthesis matrix assignments, no longer reliably demonstrated a student’s understanding of wider scholarly conversation around a topic, or even a single research article.

Elizabeth and Amanda spent a week at CAT’s Course Design Institute this summer to re-think how to better teach and assess the skills necessary to do research. We decided to let go of the trappings of the traditional, ‘finished’ research project (see our Wuthering Heights meme above!) and re-consider which elements of the research process students struggled with in isolation from one another, especially in the current AI-era. Some of these were easy for us to spot with our scaffolded research project assignments, like how to frame research questions to be open-ended and unbiased (this was discussed in more detail in our first post). Others were less obvious to us and required us to listen to our students.

In our second post, we discussed Elizabeth’s Padlet activity (that Amanda soon “borrowed”) that asked students, “what information skills do you think your generation struggles with?” In addition to feeling uncertain about what information is trustworthy, students reported struggling with “attention span,” “reading comprehension,” “reading long pieces of information,” “patience with long readings,” and “focus in the age of distraction.” It became clear after running these Padlets over several sections, that perhaps one thing we should begin focusing more of our instruction on was reading and understanding the sources that students chose. And if understanding a scholarly conversation is one of the first principles of library research (as discussed above), and if understanding a scholarly article is the what can best protect against believing misgrounded claims, what can we do if students struggle to sustain the attention that understanding requires? It became clear that we needed to make more room for reading, questioning, and making sense of sources within our assignments.

Putting the “Literacy” Back in Information Literacy

In response to these student concerns and the newly emerging, but necessary skill of being able to reconcile AI generated summaries with the actual content of written work, we decided to spend more time on is deep reading – hence, putting the “literacy” back in information literacy. Rather than submitting an annotated bibliography or synthesis matrix with many sources, students select single sources using suggested criteria, then print it out, read it, and physically mark it up (with highlighters and pens) to demonstrate a deeper understanding. In addition to physically marking the articles up, students explain their selections in a written assignment. To guide their reading, we have adapted a version of Keshav’s Triple Pass Method for reading scholarly articles. In our version, students are asked to identify the main purpose, thesis, or hypothesis of the article, supporting arguments, previous scholarship in the literature review that may help their research, the study methodology (if applicable) and passages from the discussion that help answer their research questions.

The purpose of this slower reading is to help students develop their own understanding of a piece of scholarship and its relationship to other works. Because many of the popular AI tools (including ones this university licenses, like NotebookLM) encourage students to just upload a link or PDF to get a one to two paragraph summary of a complex, lengthy article, the ability to develop a deep understanding of an original source gives students the ability to exercise their critical thinking skills and build confidence in their ability to identify bullshit (Frankfurt’s definition). Determining whether an AI-generated summary or claim accurately represents that article it purports to represent requires the ability to read and understand the original source material, which we maintain is a skill that will set students up for success in nearly every upper-division course, in graduate school, and in any career that asks them to make decisions based on evidence.

Teaching students to read scholarship more carefully has allowed us to move beyond just verifying the accuracy of AI-generated citations and evaluate the accuracy of outputs grounded in scholarly articles – whether those come from general-purpose AI tools like ChatGPT and Claude or AI research assistants licensed by the library. The shifted emphasis on more basic, careful attention to reading and understanding information particularly scholarly information, and what differentiates it from other information formats, has lent itself to more honest conversations with students about the purpose of research more generally. We think this also helps students consider where AI may be appropriate in the research process and when we should refrain from using it, whether using AI for research assistance can impede the learning process, and how information is changing in the age of AI.

In Conclusion

If you stuck with us through all three posts, thank you! It’s been a long four year (!) journey since ChatGPT 3.5 launched in Fall 2022. We’ve incrementally redesigned, tested, and redesigned, and we’re pretty happy with where we’ve landed. The underlying takeaway is that AI allows us to take shortcuts, for better or worse. Sometimes, isn’t a bad thing – we certainly won’t be sad if Boolean searching eventually go away! But it’s also often a pretty poor substitute for students’ own research skills and understanding. As AI continues to change, and more library vendors enter the market with their own AI products, we imagine LIB100 will need to continue to evolve. Whatever form it takes next, we remain committed to our core missions – helping students learn the skills, dispositions, and habits of mind required to use information thoughtfully and critically.