Title: Arizona State University Library Tutorials – PICO: Research Questions for Health Sciences
URL: https://www.asu.edu/lib/tutorials/storyline/pico/story_html5.html
Creators: Janice Hermer and Kevin Pardon
Editors: Bee Gallegos and Deirdre Kirmis
Narrator: Shane Hunt
Contributors: ASU Library, Online Tutorials & Learning (OTL) Team (Sam Dyal, Bee Gallegos, Ashley Gohr, Janice Hermer, Dennis Isbell, Lisa Kammerlocher, Deirdre Kirmis, Marc Mason, Kevin Pardon, Tammy Wolf)
Institution: Arizona State University
Interviewees: Bee Gallegos, Deirdre Kirmis, and Kevin Pardon
Interviewer: T. Eloise Stevens
Description of Project (provided by interviewees)
The PICO: Research Questions for Health Sciences tutorial is part of a series of general research skills tutorials developed for ASU students. Although the focus, as the title implies, is the health sciences, the PICO framework has value for students in other disciplines who are trying to define a topic and develop a thesis statement or answerable research question. This tutorial is licensed through Creative Commons, so individual branding and other modifications can be made with attribution.
Q: The previous two tutorials (MLA Citation Style and Academic Integrity; Site of the Month interviews available here) have been on topics that would be of interest to a wide range of departments and classes. This tutorial is more specialized to Health Science fields. How did this impact your team’s approach and creation process?
PICO is core to nursing and the health sciences, so the team saw this tutorial as a high priority following the completion of some of our general concept tutorials. Faculty in the College of Health Solutions and the College of Nursing and Health Innovation requested that this tutorial be developed, and our librarians were supportive of that need. The Online Tutorials & Learning (OTL) team actually saw the specialized nature of this tutorial as a benefit. Since the majority of our members had limited or no experience using PICO, we could look at the content from the student’s perspective. If we did not understand the content as librarians, then students likely would not understand it either.
Q: PICO is a relatively specialized concept. How did you teach yourself about PICO or collaborate with colleagues who knew about this discipline-specific tool? How do you balance competing visions of the project between tech experts and subject experts?
As a topic not usually part of an MLS/MLIS program, librarians gain knowledge and a better understanding of PICO by talking to colleagues, completing professional development courses, and then having to teach the concept to students. Our health science subject experts developed and storyboarded the content for the tutorial with some suggestions about how it could be implemented in Articulate Storyline based on what we had seen in other tutorials and our knowledge of the software. OTL members reviewed the content and provided feedback. Following that, our tech expert transferred the storyboard content into Storyline, which was reviewed again before finalizing.
Q: Is this tutorial included as part of your learning badge system? Can you elaborate on this system and how it’s used across campus?
At this point, the PICO tutorial is not included in the learning badges system, but one of the health sciences librarians has discussed a “Nursing” or “Health Sciences” badge with faculty. He was one of the original members of the learning badges team in 2013 and his subject faculty are interested in having a badge. He created a badge for a specific nursing class, prior to development of the PICO tutorial, which currently includes general concept tutorials.
Q: What is a surprising challenge that you have encountered with the Learning Badges? What advice would you give to libraries that are considering implementing a learning badge system?
Technology has always been, and continues to be, the biggest challenge we face. We looked at several platforms, but they all had problems, were not far enough along in the development process, or didn’t integrate well with the LMS. We choose a WordPress platform for our learning badges, which still has had its challenges. We are on the second quiz plugin and will be changing to a different one soon because of technical issues related to communicating with Credly and the awarding of certificates/badges. Our advice would be to test a variety of products and how they interact with other systems and products. Be patient and understand that it isn’t a quick and easy process. On the non-technical side, we recommend that in addition to marketing to faculty, you should also reach out to your librarian colleagues who can often be the toughest audience.
Q: Which populations/classes use this tutorial (undergrad/grad/ majors in and outside of health sciences)? With such a large student body, and so many programs, how do you decide which subject areas to develop tutorials for?
Primarily, the tutorial is used by and intended for health sciences and nursing students. The School for the Science of Healthcare Delivery, for example, uses this tutorial as an assigned tutorial in their capstone course. It will also likely be implemented by the online nursing program when we update tutorials for the spring semester. Within the health sciences, we do have to prioritize certain programs. Typically programs that are larger and/or more active as far as using the library (faculty, students, etc.) or have a large number of online-only programs are the ones we would focus on developing or implementing online modules for. Several physical education faculty also like to use the PICO model. It is also used by graduate students across disciplines outside of education, such as social work, recreation, the health sciences, and sports, depending on their thesis topic.
Q: I liked the framing and scaffolding of your outcomes (e.g., ‘Explain what PICO is and what the letters stand for,’ ‘Apply PICO framework to a specific topic or scenario,’ ‘Formulate an answerable research question using PICO’). Can you explain your process for developing learning outcomes and what role they play in the design process?
Much of our process for developing the learning outcomes comes from our experience teaching PICO in an in-person setting. In these courses, we followed a similar process of introducing and defining the concept, practicing with the class using a scenario or two, and then turning that scenario into the actual research question (and then having time for hands-on practice with students). This method seemed to work well so we tried to adapt it to the online environment. To develop the learning outcomes we asked ourselves what the key skills or pieces of knowledge we wanted students to have after completing this tutorial were. The key purpose of using PICO is to be able to develop a research question, so we knew that would be our final learning outcome. It can be difficult to just jump right into building a research question, though, which is why we tried to scaffold the learning outcomes to build up to that final goal. The learning outcomes were vital to the design process, though, as they added structure to our content and made it easier for us to keep things simple and not get lost overthinking things or making the module too complex.
Q: I found the part about turning a PICO question into researchable keywords particularly valuable. It also seems like it could be tied into other library and research services. How do you coordinate with subject instruction librarians to create instructional experiences that meet students where they are?
The PICO model is similar to others that analyze a topic and work toward a research question, so it is, in fact, being used under different labels. The OTL team is comprised of a cross-section of librarians supporting a variety of disciplines in the humanities, social sciences, and sciences and located on three of the five Phoenix metropolitan campuses, so we have a good sense of the subject needs and local campus environments. Additionally, OTL has a form available to librarians for submission of tutorial ideas. Librarians can choose their level of involvement in developing new tutorials. This involvement may include content suggestions, content development, and, depending on technical skill level, may include developing the product with assistance from OTL. Each option includes review by OTL members at each level of the tutorial development.
Q: What are your legal responsibilities for making tutorials accessible, and how do you meet or exceed them? What do you find most difficult or time-consuming about adhering to accessibility standards?
We strive to comply with the WCAG 2.0 standards for accessibility in our tutorials, which includes elements such as keyboard functionality, text alternatives for narration and non-text content, multiple navigation paths, predictable content and navigation, and good color contrast. We have provided two alternative transcripts for the narration, and we have enforced standards in layout, structure, color scheme, and typography of all tutorials. The most difficult issue involving compliance has been the identification and understanding of each standard and how it relates to online content, and how to best deliver content that follows each standard. Some items in interactive tutorials can be challenging, such as slide transitions and activities with user input, and require that we think carefully about the content as experienced by students who may need assistance. Now that Storyline 360 includes a closed-caption editor, we plan to implement this feature in our tutorials by spring 2018.
Q: In September’s interview, you mentioned that “Content creation has never been a problem with a core group of 3-5 individuals being involved from the beginning with the learning badges project; the ongoing challenge is and has always been lack of technical support.” If you could create a new position for someone who would be most valuable to your team, what would that position description look like? What technical skillset would you like to see added?
The OTL team is comprised primarily of librarians, thus the comment about content development. The Learning Badge team, which preceded OTL, was on hiatus for approximately a year due to the lack of technical knowledge or support. We had faculty buy-in and some already-created tutorials but lacked an environment in which to make them accessible. We had decided to add them to the general tutorials page when library administration came to our rescue. The Head of Discovery Services developed the WordPress platform we use for badges and was put “on loan” to us for a short time to help us get back in business. In spring 2015, the library hired a web application developer; she supports many services and products but has enabled us to get back on track with the badges and tutorials. She is the primary technical support person and is working on a graduate degree in instructional design. If possible, we would love to clone her! Without her technical expertise and support, tutorials would likely still be on our wish list. Given her other duties, she is pulled in multiple directions. We would literally like to have someone who is just like her: an instructional designer with technical expertise in web products and the development of instructional materials.
Q: Since quiz data for the tutorials has not been analyzed yet (as you mentioned in last month’s interview), can you elaborate on how you revise tutorials based on assessment data? What does that process look like?
The Action Lab, a digital learning research center, provides unique, multidimensional, and actionable research on advanced teaching and learning technologies. They are interested in collaborating with OTL and the ASU Library to analyze use of the tutorials. We are in the beginning stages of this project.
