ACRL announces the publication of Assessment and Advocacy: Using Project Outcome for Academic Libraries, edited by Gena Parsons-Diamond, demonstrating how a variety of libraries have used Project Outcome to make improvements in their practice and highlighting the value the toolkit has brought to institutions and the academic library profession. Proceeds from the book go to keeping Project Outcome free.
Learn more about Assessment and Advocacy in this excerpt from the Introduction by the editor, licensed under CC BY 4.0.
What is Project Outcome?
While many libraries collect data about their programs and services, it can be more difficult gathering evidence that captures the benefits libraries provide to students and other users. Measuring outcomes can provide libraries with new ways to demonstrate their effectiveness beyond gate counts and anecdotal success stories.
Project Outcome is designed to help libraries understand and share the impact of essential library programs, instruction, and services by providing simple surveys and an easy-to-use process for measuring and analyzing outcomes. ACRL believes so strongly in the value of this tool for the profession, and places such a high importance on libraries implementing outcomes-based assessment and improvement, that the association invests $60,000 annually in making the tool freely available. This support ensures the tool is available to all post-secondary libraries without a subscription fee.
Project Outcome provides libraries with access to quick and simple patron surveys, an easy-to-use survey management tool, custom reports and interactive data dashboards for analyzing the data, and various resources to help libraries implement the surveys and then use the results. Libraries are encouraged to use their results to support and promote future action—allocating resources more efficiently, advocating for new resources more effectively, and providing support for future library funding, activity reports, and strategic planning. Whether new to outcomes measurement or advanced in data collection, all academic libraries can access the standardized surveys, national and peer benchmarking, and data analysis tools to effect change within their institutions and beyond.
Project Outcome’s standardized surveys allow libraries to aggregate their outcomes data and analyze trends by service topic, program type, and over time. Academic libraries can see how the outcomes of their programs and services compare across their institution, Carnegie classification, and nation. Project Outcome for Academic Libraries surveys help libraries measure outcomes and assess their impact in seven key service areas (see Figure 1).
Figure 1. The seven survey topics offered in the Project Outcome for Academic Libraries toolkit.
The outcomes measured in the Project Outcome surveys are one piece of the assessment puzzle. They help libraries understand the specific benefits that result from their services or programs. Outcomes can be quantitative or qualitative and are often expressed as changes that individuals perceive in themselves. They answer the question: “What good did we do?” (See Figure 2.)
Figure 2. Four key outcomes measured in the Project Outcome for Academic Libraries toolkit.
Project Outcome provides three types of tools for libraries to measure their patron outcomes (see Figure 3).
Figure 3. The three types of tools provided in Project Outcome and what they measure.
Each Project Outcome immediate survey is six questions long and includes both Likert-scale and open-ended questions. Libraries can add up to three custom questions to help them gather additional evidence. The immediate surveys are designed to be distributed right after a program or service is completed and aim to help libraries better understand the immediate impact that program or service had on patrons and their intention to change their behavior as a result. Results from immediate surveys are ideal for informing program or service changes and providing a snapshot for advocacy and reporting. The standard immediate and follow-up surveys for all topics are available in Appendices C and D at the end of the book.
Each Project Outcome follow-up survey has five questions and follows a Yes/No/Not Applicable/Please Explain format. The follow-up surveys are designed to be used four to eight weeks after a program or service is completed and aim to help libraries better understand if patrons have changed their behavior or continued to benefit as a result of the program or service. Results from follow-up surveys are ideal for assessing the lasting impact of a program or service, informing internal planning, measuring progress toward strategic goals, and providing evidence for advocacy.
The Outcome Measurement Guidelines[i] included in the toolkit resources provide additional support for outcomes-focused data collection. The guidelines focus on four areas: developing outcome measures, alternative data collections methods, measuring outcome data over time, and working with partners. They, along with the many other resources, are designed to help libraries conduct advanced methods of outcomes measurement and demonstrate long-term, collaborative impact on their users and institutions.
User Engagement
Between the April 2019 launch and July 12, 2023, 5,016 users (librarians, library workers, and LIS students) have registered for the Project Outcome for Academic Libraries toolkit. The users come from 1,584 institutions in 65 countries. There have been 12,154 surveys created in the toolkit that have collected 159,252 responses. See Appendix E for a full list of institutions who have collected survey responses. While Project Outcome is open to all academic and research libraries globally, the majority of registered users (over 76 percent) and survey responses (96 percent) come from institutions in the United States.
Surveys are distributed and responses collected by institutions from all Carnegie classifications. Community colleges and 2-year institutions account for 17 percent of response data, four-year institutions for 16 percent, master’s-granting colleges and universities for 33 percent, doctoral-granting universities for 31 percent, and special focus institutions for 3 percent. As shown in Figure 4, Associate’s Colleges are underrepresented in Project Outcome data, while Master’s Colleges & Universities and Doctoral Universities are overrepresented.
Figure 4. Chart comparing Project Outcome usage by Carnegie Class to overall institutions by Carnegie Class.
Organization of Assessment and Advocacy
This book begins with the chapter Characteristics of Academic Libraries, Library Instruction, and Student Outcomes: A Descriptive Analysis Focusing on Minority-Serving and Rural Institutions by Kara Malenfant and Sara Goek. Malenfant and Goek discuss several national-level academic library survey tools and datasets and how the Project Outcome data can be used in conversation with these sources. Their ultimate goal is to encourage greater participation in national-level surveys and using this data for future research opportunities.
Following this are ten case studies, each of which describes how use of the Project Outcome for Academic Libraries toolkit has benefitted the authors’ library and institution.
- Three Newly Joined Libraries Seeking to Better Serve Students and Grow Advocacy Efforts by Joe Pirillo
- The University of Wisconsin Oshkosh (UWO) aimed to assess the impact of library support on student success, particularly in Writing 188 classes, to contribute to a new library strategic plan. Using Project Outcome surveys, the university’s instruction librarians gathered feedback from students and individual research appointment participants, leading to workflow adjustments and insights into possible future improvements for the library.
- Collecting Iowa State University Library Spaces Feedback Using Project Outcome by Greg Davis
- Iowa State University (ISU) uses the Project Outcome space survey to assess library spaces and make data-driven decisions. The survey, conducted semi-annually since fall 2018, involves collecting both quantitative and qualitative data from students who have reserved library study rooms.
- Using the Instruction Survey as Part of a Three-Tiered Approach to Information Literacy Instruction Assessment by Karoline Manny
- The Centre College library’s information literacy instruction program has used Project Outcome instruction surveys since the 2020-2021 academic year as part of its three-tiered assessment plan: course-level, satisfaction-based assessment; competency-based assessment; and program-level, competency-based assessment. The goal of this assessment is to provide student feedback to library instructors and use the results to demonstrate the value of library instruction to faculty and administrators.
- Two Academic Libraries, One Study: Assessment and Advocacy across Partner Institutions by Katy Miller, Rachel Trnka, and Karen Kaufmann
- Librarians from two institutions, Seminole State College of Florida (SSC) and University of Central Florida (ICF), collaborated on a project to study library instruction outcomes across both campuses using the Project Outcome instruction survey. The resulting data was used to align information literacy instruction at SSC and UCS, strengthen their partnership, and support student success.
- Small Changes, Big Impact: Assessing Group Study Rooms Using the Space Survey by Jennifer Arnold, Christy Case, and Retha Hall
- In July 2022, Central Piedmont Community College opened their new library, the largest library space of their six campuses. Staff used the Project Outcome space survey to assess the study rooms as a part a larger assessment plan, considering in particular potential changes to the study room configurations that could improve student knowledge, confidence, and awareness of library resources.
- Revitalizing our Instructional Assessment: Embracing a New Approach Using Project Outcome by Martha Hood and Juliana Espinosa
- After two years of virtual instruction due to the pandemic, the librarians at the University of Houston-Clear Lake undertook the task of updating and modifying the first-year experience course. They used the immediate instruction survey to gather a clear vision of areas which needed improvement and used responses to promote the value of the library’s instruction program to faculty and other stakeholders.
- Using Instruction Surveys to Establish Baseline Data for Library Instruction Improvement by Laura D’Aveta
- Before January 2022, the library at Hiram College had not collected any assessment or performance data about library instruction sessions. Following an administrative decision that made library instruction a mandatory part of the college’s first-year seminar courses , they used the instruction survey to determine if the mandate resulted in an improvement in student performance.
- Using the Instruction Follow-Up Survey to Advance Library Integration in the First-Year Experience Program by Courtney Drysdale and Paul Betty
- Regis University used the follow-up instruction survey to build a stronger argument in favor of mandatory library instruction for all sections of the first-year experience course at their institution.
- Using the Instruction Survey for an Undergraduate Library Orientation by Kaitlyn Clohosey
- The research and instructional services librarian at Felician University used the immediate instruction survey in the library orientation session for all first-year, first-semester students to yield data on how well students remembered information about the session and if their library awareness increased. The resulting data was also included in end-of-year reporting and to advocate for the permanency of a mandatory library session.
- Assessing Information Literacy Programs for First-Year Writing Courses at a Public University by Kate Langan, LuMarie Guth, and Dylan Juhl
- Librarians at Western Michigan University used the immediate instruction survey to assess the effectiveness of library programming for first-year writing courses in different colleges at a mid-sized public university. They used the results to assess modes of instruction, learning outcomes, and student affect.
Conclusion
Project Outcome provides valuable data, benchmarking, and feedback from students, faculty, and other library stakeholders, which can help you and your library advocate for continued investment, improve instruction, strengthen partnerships, and much more. Register for free today at https://acrl.projectoutcome.org and let us know how you’ve implemented the ideas in this book and your new ideas to effect change at your library, institution, and in the profession.
[i] Project Outcome for Academic Libraries, “Outcome Measurement Guidelines,” June 2023, https://acrl.projectoutcome.org/surveys-resources/outcome-measurement-guidelines.