What’s Behind a Web Search? Bias and Algorithms
https://jmu.libwizard.com/f/algorithms-bias
Creator and Interviewees: Carolyn Schubert, Malia Willey, and Alyssa Young
Institution: James Madison University
Interviewer: Amanda Kaufman
Description (Creator Provided): While Google is a common tool used in research, how much do people understand about how Google does and doesn’t work? This tutorial aims to provide the broader context about search algorithms, their creation, and their limitations. The tutorial focuses on Google’s search algorithm as an example to see these contextual elements in practice. We conclude with opportunities to reflect on potential algorithmic biases and identify possible actions for countering bias. We developed this asynchronous, interactive tutorial for first-year and transfer students, but it can be useful for anyone new to the topic, including faculty and graduate students.
Q: What inspired the creation of the What’s Behind a Web Search? Bias and Algorithms tutorial?
A: A few years ago, Carolyn read Safiya Nobel’s book Algorithms of Oppression: How Search Engines Reinforce Racism and was inspired to incorporate this work into our information literacy instruction about searching. As liaison to health profession and nursing programs, Carolyn was also seeing more and more research pop up related to algorithmic bias in health care records and decision support systems. During COVID, regular instruction and consultation work was a bit in flux, so we had a little more capacity to think about new workshop ideas. Carolyn decided to approach colleagues to see if anyone else had interest in working together. Valuing the different perspectives, experiences, and disciplinary backgrounds we can collectively contribute provides a good check on individual assumptions. Alyssa and Malia volunteered to collaborate. We were inspired to build upon the Exploring Algorithmic Bias with a Summer Bridge Program activity Elisa Acosta shared through the open access resource CORA (Community of Online Research Assignments).
Q: Could you tell us more about the team of people who helped create the tutorial and their individual roles?
A: We are members of the Research & Education Services department, which is the home of our liaison librarians. As liaisons, we teach students how to navigate and evaluate information. Carolyn Schubert is the Director of Research & Education Services team and liaison to the School of Nursing. Malia Willey is a Humanities Librarian and liaison to History, Philosophy & Religion, and Sociology & Anthropology. Alyssa Young is a Science & Math Librarian and liaison to Biology, Chemistry, Geology, Mathematics & Statistics, and Physics & Astronomy. We all equally worked together on this project.
Q: Who is the intended audience, and what is the primary intended use of the What’s Behind a Web Search? Bias and Algorithms tutorial?
A: The intended audience was General Education students, but we also wanted to have flexibility to adapt the content to other populations and to make this shareable to other institutions. The intended use is to bring awareness of algorithms and how search engines can amplify algorithmic bias.
Q: What technologies and software did you use to create this tutorial? Was there anything new you had to learn? Can you tell us a little more about that decision making process?
A: The tutorial involved a few different pieces of technology. We collaborated on an outline, scripting, and question development through Microsoft Word via OneDrive. We individually created our personal video tutorials through Camtasia and uploaded those materials to YouTube. To bring the written and video content together, as well as the formative assessment, we used LibWizard. Carolyn had some prior experience with LibWizard, but this was new for Alyssa and Malia, making this an opportunity for them to learn this new tool as well.
Q: I like that you included a mix of self-created content with freely available content (e.g. YouTube videos) in the tutorial. Could you tell us more about your decision process about whether to generate all-new content from scratch or use something already created?
A: There are some great talks by the experts themselves, so it made more sense to elevate their voices rather than parrot their excellent work. We also wanted to avoid investing time in replicating good content that already exists. However, to make the narrative flow smoothly, we created custom pieces to tie the tutorial together.
Q: Creating tutorials can be a very time-consuming process. Can you describe for us the timeline you went through for conceptualizing, developing, and launching this tutorial?
A: We started in early/mid Fall 2020. We met periodically to review progress, give each other feedback, and set our next steps in the work. In early Spring, we asked colleagues within the Libraries to pilot test the tutorial. We completed revisions and launched the tutorial for students in early March 2021.
Q: Ensuring that tutorials are fully accessible can be challenging. What was your process with regard to accessibility?
A: We all value and commit to making instructional materials as accessible as possible, so we had this as a foundational value from the beginning of our planning. Given our experiences with online teaching in COVID, we had already become familiar with captioning and creating transcripts for video tutorials. We included alternate text for images. Malia’s experience with Universal Design for Learning also greatly informed the project’s development.
Q: How are you promoting the Algorithms tutorial to your university community?
A: Initially, we launched this tutorial in Spring 2021 when many classes were still online or hybrid. We offered this tutorial through the Passport event series for our General Education curriculum. There is a requirement for undergraduate students to participate in a selection of events. Students could complete our tutorial online and asynchronously.
Starting in Fall 2021, the General Education program has shifted back to only having Passport events be in-person, so the audience has changed. Instead, Carolyn has individually promoted this as a resource to integrate into courses. One avenue was incorporating this content into an introductory news writing and reporting class to support journalistic information literacy skill development. Additionally, the Surgeon General issued a report in Summer 2021 about the issues of misinformation. As part of that report, he included explicit calls to action for health profession educators to talk about algorithmic bias. Carolyn has been able to easily justify expanding this content as part of existing online modules for graduate and doctoral nursing research courses. Alyssa currently facilitates discussions or assignments on data ethics in some of her science instruction when introducing data management practices. She plans to integrate this resource into some introductory courses and remix the content for an upper-level course.
Q: There are a mix of multiple choice questions and fill-in-the-blank questions on the tutorial. Can you tell us more about the built-in assessments and how you plan to use them?
A: The goal of the tutorial was not necessarily to provide summative assessment, but more small bits of formative, self-driven assessment and reflection along the way. For the General Education event series, students write their own assessment of the event so we did not need to prioritize further reporting out in this first iteration. We were surprised to see over 300 people took the quiz between March and May 2021. We plan to review this data for internal assessment and tutorial refinement. As we look to integrate the tutorial into other courses, we’ve discussed pursuing an IRB proposal to conduct further research and publish.
Q: Can you talk about your process or plan for revising and keeping the tutorial updated?
A: Our General Education program is undergoing academic program review at the moment. We currently have a set of video tutorials called the Madison Research Essential Toolkit that all incoming students take through General Education coursework. As we hear back about the program review, we are hoping to see an opportunity to integrate this tutorial into the Toolkit or in other parts of the general education curriculum. Librarians from other institutions have expressed wanting to create something similar. We shared our materials through CORA so others can adapt the tutorial for their needs.
