PRIMO Site of the Month: April 2017
Title: Scholarship Is a Conversation
URL : https://www.softchalkcloud.com/lesson/serve/puLXxFYUW8EKkH/html
Authors: Julie Hartwell, Geoff Iverson, Sara Kearns, Kate Otto, Ashley Stark, Rachel Vukas
Institution: New Literacies Alliance
Interviewee: Joelle Pitts
Interviewer: Marcia Rapchak
Tutorial Description (provided by the authors): The New Literacies Alliance is an inter-institutional consortium of academic libraries aimed at building ACRL Framework-based online information literacy lessons. The lessons can be embedded in websites and LibGuides, synced with most learning management systems, or hosted in the cloud for students to review. All lessons are institution-, vendor-, and technology-agnostic—meaning that they can be used by any institution. The lessons are also licensed through Creative Commons, so individual branding and other modifications can be made. This lesson is mapped to the Scholarship as Conversation Frame and introduces the concept of scholarly conversations developing over time, and how to follow a scholarly conversation.
Q: Last month, you discussed who was involved in the New Literacies Alliance. Who was involved in the Scholarship is a Conversation lesson particularly?
A: Sara Kearns, Ashley Stark, Alice Anderson, and Ashley Flinn from Kansas State University; Julie Hartwell, Rachel Vukas, and Heather Healy from University of Kansas Medical Center; Geoff Iverson from University of Wisconsin-Platteville and Kate Otto from Indiana University at Bloomington worked on this module.
Q: What did you learn from your rapid prototyping design and development process for this lesson? What changes did you make based on feedback?
A: Students provide feedback for each prototype. Each test is one-to-one—one interviewer and one student. While the student takes the lesson, we record the screen and audio. This is followed by a semi-structured interview to explore their perceptions and opinions regarding the lesson. The feedback from each student test is compiled and analyzed by the lesson small group and used to create the next iteration or prototype. Usually two or three iterations follow until we reach a “saturation” point where we don’t hear much constructive feedback from the students. At that point the lesson goes through an accessibility check and a copy edit process.
We learned many things from each prototype effort, so let us examine just one. This lesson started normally, deciding on an objective, and then drafting content and an assessment. After our first draft, we found our content outstripping the objective by quite a bit. We had some content for citation mapping, other content for argumentation. Each of these is part of the scholarly conversation; however, the content lacked a cohesive story arc and so was dull. It was akin to telling the story of the Three Little Pigs by saying the pigs had three houses. The wolf destroyed two.
After revisiting the original objective that had seemed plausible and after many discussions, we found it was not the real objective, not a good first-thing-to-learn objective for our first lesson in this area of the framework, so we changed it. Instead of focusing on what a scholarly conversation is, we examined what to do to engage in a scholarly conversation.
Our experience with the prototype also identified opportunities for many more lessons on scholarship as conversation.
Q: What was involved in the development of the timeline? What software was used to create it? What made you decide to use a timeline to represent scholarship as a conversation?
A: After the first prototype, we had several meetings about the essential nature of scholarly conversations. How are they the same and how are they different from everyday conversations? What do students not fathom? We wanted students to learn via active learning, but how could they experience the rich complexity of a genuine conversation in under 15 minutes?! In these meetings, the group talked a lot about scholarship over time. Timeline software enables you to digitally represent time.
However, timelines give a strong linear impression, a potential problem for representing scholarship. A timeline of scholarship is not simply “this was written and then that was written” – that would be a citation map. Scholars do not read three articles and write an essay on Friday. Scholarship is more; it is a mesh of activities, a set of behaviors. We needed to identify a collection of strategies that our students could employ and we needed scholars to model those strategies. The challenge was to illustrate the activities of the scholars rather than to tell the history of a subject like PTSD.
Developing the story for the timeline involved researching the historical development of the topic, PTSD, which is long and rich with big paradigm shifts and is interdisciplinary. Sara Kearns had already collected a large portion of the research through her work with a professor. We also researched scholars and chose to feature one, mentioned another, and created two student-scholar characters, as we wanted to represent reality as well as peer-models for the students. This research helped us identify scholarly conversation activities, which we inserted into the timeline text and identified in bold or italics.
We constructed the timeline using the TimelineJS software made by Northwestern University Knight Lab. It is open source and available at https://timeline.knightlab.com. Several public domain images were located on Wikimedia Commons, some of which we adjusted using PhotoShop.
Q: How long was the creation process for this module from initial brainstorming to finished product?
A: This module took part-time work over two semesters.This part of the Framework, Scholarship as Conversation, feels new. Unlike some other areas of the Framework, there were not dozens of example approaches to examine. For example, the disposition to recognize the ongoing nature of scholarly conversations is not an objective that gets coverage in a typical introductory class. As mentioned earlier, we experienced a false start on objective/content alignment. We were searching for a way to allow students to see what went into making a scholarly conversation.
Q: How have you used the assessment data from the module so far? Do you have any other plans for how you will use the assessment data?
A: The lesson is currently being used by several faculty members at various institutions and we will first perform a question validity analysis once we get the score data from the instructors. Question response data is gathered during each prototype and when instructors choose to share the score results with us after their classes have taken the lessons. Question validity analysis is a study of the validity of the assessment questions used on a particular assignment or activity. Take the number of correct answers divided by the total number of answers and you get a percentage. Percentages over 50% are typically considered valid (according to the KSU Education Department), but many instructors choose a higher threshold (say, 75% correct) to consider their questions valid. Since these lessons are intended to be used asynchronously, we have chosen a relatively high threshold of 80% correct answers to denote a valid question. Any questions with 79% and below correct answers are reviewed and modified. This lesson will also be included in upcoming qualitative assessment in the form of faculty and student focus groups aimed at determining the effectiveness of NLA lessons in their courses and studies, respectively.
Q: What modules will the New Literacies Alliance create in the future? How do you plan to market and share your lessons?
A: We are nearly done with a lesson on evidence-based practice in the research process. We have just begun work on a lesson on evaluating information on the web, specifically examining the process how one might perform an initial evaluation of a source and also touching on the cognitive processes that we can employ to break out of our own filter bubbles. Starting in April, a small group will begin creation of a lesson on how to digest/read a research article. All new lessons will be released on our new website newliteraciesalliance.org, and will be submitted to various repositories such as PRIMO, CORA, and the ACRL Framework Sandbox. We are also going to create a space within Canvas Commons as a way for instructors and librarians using Canvas to pull the lessons into their courses more easily. As usual, all new lessons will be licensed via Creative Commons and anyone will be able to use and modify them to fit the needs of their course or instruction program.
Q: Last month you mentioned that NLA is creating a design institute. Could you tell us more about that and how interested librarians could become involved?
A: We are in the process of requesting funding to create an intensive design institute for librarians who want to learn more about instructional design and lesson creation using the NLA Project Foundation. If funded, it will likely take place in Kansas City in 2018. We don’t anticipate we would plan an institute unless we receive some extramural funding. Participants will spend several days creating learning objectives, brainstorming content, writing assessment questions and plans, and working directly with instructional designers to begin adapting their content online. Our hope is that participants will come away with a deeper understanding of instructional design principles, how they might continue to utilize their new skills at their institution, and with the beginnings of an NLA lesson that could be utilized both at their institution and within the suite of NLA lesson offerings. Interested librarians can contact one of the NLA Steering Committee members (Joelle Pitts, jopitts @ k-state . edu, Heather Collins, hcollins @ kumc . edu, or Matt Upson, matthew . upson @ okstate . edu) to be put on a mailing list where we will send news and updates.
Q: Any other comments or advice you want to share?
A: We love feedback and are very interested in hearing from any instructors or librarians who have used our lessons to help us make them better. If anyone is interested in participating as a content creator or instructional designer, or would just like to know more about the project, they can contact us at any time at jopitts @ ksu . edu.
Note: Joelle Pitts, the project author interviewed here, gave a webinar for ACRL’s Distance Learning Section where further details regarding the use of collaborative instructional design to create online learning experiences. The recording of the webinar, Collaborative Instructional Design: Leveraging Resources to Build Online Learning Experiences, can be found at https://youtu.be/yxr0KUvgpKs.
