Orr, R. H. (1973). Measuring the goodness of library services: a general framework for considering quantitative measures. Journal of Documentation, 29 (3), 315-332.


While Orr’s model (depicted above) has not been utilized in any studies published in the last two decades, it is one of the earliest academic library evaluation models and it provides a useful way of conceptualizing different definitions of library value. It is essentially a systems based evaluation model in that inputs (resources) are linked to processes, which are then linked to outputs (beneficial effects). However, it also incorporates a library’s capability to provide services, its users’ demands for service, and how user utilize these services. Orr’s model is useful for library value research because it separates the concepts of quality, which is related to the library’s capability to provide resources and services, and value, which is related to the library’s beneficial effects on its users and parent institution. Because of this distinction, Orr was one of the first to identify that library goodness/value varied depending on whether the library’s services and resources are being judged from the user or institutional perspective. Users would be interested in library quality, but would be less concerned with the financial value of the library.


Small, R. V., & Snyder, J. (2010). Research instruments for measuring the impact of school libraries on student achievement and motivation. School Libraries Worldwide, 16(1), 61.

This study focused on school libraries in the state of New York rather than collge or university academic libraries. However, the topic, methodology, and research instruments developed in the study may also be applicable for similar studies in academic libraries. Other papers on the research findings from the study have also been published.

The purpose of the study was to develop research instruments for measing the impact of libraries on student achievement, motivation for learning, and technology use. It consisted of three phases. The first was an online survey sent to librarians and principals in the state of New York; the second was an in-depth survey sent to librarians, teachers, students, principals in 47 schools in New York; and the third included focus groups and nterviews with librarians, teachers, students, principals, and parents at 10 schools, plus observations of libraries and interviews at 2 schools. Overall over 3700 people participated in the study.

In addition to the qualitative methods, quantitative methods including factor analyses were also utilized in order to test the reliability of the instruments. The resulting instruments include scales for information literacy (one each for finding, using, and evaluating information), technology (importance of the librarian teaching computer skills), respect for diversity, collaboration, professional development (for the librarian), IEPs, assistive technology, inclusion & collection development, and learning climate (an adaptation of a scale by Deci & Ryan (2008)).


Mengel, E., & Lewis, V. (2012). Collaborative assessment: North American academic libraries’ experiences using the Balanced Scorecard to measure performance and show value. Library Management, 33(6/7), 357-364.

This study reviewed balanced scorecard performance measures from Johns Hopkins University and McMaster University. The researchers were interested in how many measures overlapped, how many of the measures used data from ARL’s annual statistics, and what tools could help create collaborative scorecards, which might allow for interinstitutional comparisons. The study authors suggested that the latter was possible because libraries do similar work to support similar missions.

The study found that there were 94 measures between the two institutions. Most were customer related (41), an equal number dealt with financial or internal goals (19), and there were also several for learning and growth (15). Of these measures, there was a 9.5% overlap with the ARL statistics. The study authors concluded that creating an inventory or common set of measures. Doing so would save the time of institutions in trying to develop and test their own measures, it would allow for benchmarking with other institutions, and it would make the performance measures more credible.


Volentine, R., & Tenopir, C. (2013). Value of academic reading and value of the library in academics’ own words. In Aslib Proceedings, 65 (4), 425-440. Emerald Group Publishing Limited.

This qualitative study investigated how 2000 academic staff members from 6 universities in the UK conceptualized the value of academic reading and academic libraries. The participants were asked via survey to describe their last academic reading in terms of its purpose, the outcomes of the reading, and the value of the reading and access to library collections.

The major study findings were that articles were important for all types of academic work in various disciplines. Consequently library e-journal collections are highly valued by the participants, but electronic access was often an issue. Book collections were also valued. Although similar findings have been found in other studies, the study authors state that qualitative studies, which allow participants to describe value in their own words, can be more impactful when demonstrating library value.


Ennis, D., Medaille, A., Lambert, T., Kelley, R., & Harris Jr, F. C. (2013). A comparison of academic libraries: an analysis using a self-organizing map. Performance Measurement and Metrics, 14(2), 118-131.

This study used a self-organizing map (SOM) to investigate the relationship among resources measures and service usage. A SOM is a type of learning algorithm that uses cluster analysis to analyze, classify, and visualize data from large data sets. The study used ACRL Metrics data from 1395 US and Canadian libraries. A library performance metric (LPM), which was based on circulation, instruction session attendance, and reference transactions was correlated with expenditure, personnel, material, and service offering data.

The study findings split libraries up into four groups with combinations of high or low resources and high or low LPMs. The major study finding was that libraries with lower resources could have high LPMs if they offered more service features by hosting more presentations to groups, being open more hours, and staffing more service points. In other words, by offering more of these types of services libraries could raise their LPMs more effectively.


Lange, J., Lannon, A., & McKinnon, D. (2014). The Measureable Effects of Closing a Branch Library: Circulation, Instruction, and Service Perception. portal: Libraries and the Academy, 14(4), 633-651.

This study studied the effects of closing a branch library at McGill University. The researchers hypothesized that the closure would result in a decline in circulation and information literacy instruction. The data used in the study included circulation statistics for initial loans, number of in-class library workshops taught, and LibQUAL+ comments from Jan 2009 through Dec 2012.

The major study findings were that only the number of information literacy classes held after the branch closure were significantly lower. While circulation statistics were lower, they were not significantly so. The LibQUAL+ adequacy gaps in all three areas (affect of service, information control, and library as place) were not significnatly lower. As noted by the study authors, each of these data sources considered separately would probably not point to a loss in the library’s value by closing the branch, but the use of multiple data sources helps paint a more comprehensive picture of the closure’s effect on the library’s value.


Pan, D., Wiersma, G., Williams, L., & Fong, Y. S. (2013). More Than a Number: Unexpected Benefits of Return on Investment Analysis. The Journal of Academic Librarianship, 39(6), 566-572.

This ROI study was conducted from 2010-2011 at the University of Colorado. Faculty members from different campuses were surveyed about the articles they had cited and read for their most recently published article. In particular they were asked how many there were and how they were obtained. Feedback on library resources and services was also requested. This information was then put into the University of Colorado’s ROI model.

The major study finding was that ROI results varied significantly from -10% to 144% from campus to campus and year to year. Articles with a lower price to view (~$31.94) had ROIs of 55% while articles with a higher price to view (~$36.51) had ROIs of 77%. As the title alludes to, a few other benefits of the library’s resources and services that the study found were that there need to be increased awareness of user resource needs, that library collections and services must be promoted more, and that alternative acquisition models should be investigated.

Kingma, B., & McClure, K. (2015). Lib-value: Values, outcomes, and return on investment of academic libraries, phase III: ROI of the Syracuse University Library. College & Research Libraries, 76(1), 63-80.

This study reports the results of the Lib-Value Project, Phase III. The Lib-Value project is funded by IMLS and considers library value from economic, environmental, and social perspectives. The first phase of the Lib-Value Project consisted of one study in 2006 at UIUC that found an ROI of $4.38 for every $1 spent on the library according to budget data, faculty grant income data, and faculty surveys. The second phase consisted of a study in 2008 at 8 institutions in 8 countries that found an ROI of more than $1 for every $1 spent on the library in 6/8 locations according to institutional demographic data, faculty surveys, and administrator interviews. The third phase study took place at 3 institutions and began in 2009.

This study utilized a survey issued from Fall 2010 through Spring 2011 distributed in 26 classrooms. 782 students responded to the questions about their library usage and willingness to pay, which is part of contingent valuation methodology. Other demographic information about the students and institution were also collected. The main study finding was that the ROI was $4.13 for every $1 spent on the library.


Booth, C., Lowe, M.S., Tagge, N. & Stone, S.M. (2015). Degrees of Impact: Analyzing the Effects of Progressive Librarian Course Collaborations on Student Performance. College and Research Libraries. (In Press)

This study at the Claremont Colleges Library (CCL) studied the progressive effects of increased librarian involvement in instruction and syllabus/assignment design. Using a mixed methods design, a random sample of 99 final research papers from 13 First Year seminar classes underwent assessment using a CCL student work rubric that assessed their level of achievement in three areas of information literacy: attribution, evaluation of sources, and communication of evidence. The assessment scores were then correlated with the level of librarian engagement in class instruction and syllabus/assignment design. 3 example cases are provided to explain the engagement levels in more detail. A secondary analysis was conducted comparing faculty assessment of the librarian’s teaching effectiveness and students’ self-perceived learning.

The major study finding was that increasing levels of course instruction and syllabus/assessment design were significantly correlated with increasing levels of positive impact on student work according to the rubric assessment. Interestingly, there were no significant correlations between rubric assessed student work and librarian teaching evaluations, or between the assessed work and self-perceived student learning gains. This secondary finding caused the study authors to suggest that assessments of library teaching effectiveness and self-perceived learning statements might be less effective in demonstrating librarian impact than other measures.


Tenopir, C. et. al., (2010). University Investment in the Library, Phase II: An International Study of the Library’s Value to the Grants Process. [White paper]. Retrieved from Elsevier: http://libraryconnect.elsevier.com/sites/default/files/2010-06-whitepaper-roi2_0.pdf

In this white paper, Tenopir and a group of researchers study eight institutions in eight different countries using  a model for measuring return on investment (ROI) previously established at the University of Illinois at Urbana Champaign in order to quantify the “library’s contributions to the institution’s income-generating functions and demonstrate the economic value of investment in the library.” Tenopir et al. provide thorough documentation of related research and models of ROI before delving into data results and analysis. There is careful emphasis that ROI in the grants process is just one measurement and “underestimates the total value of the library.” The study had six key findings:

  1. For every monetary unit invested in academic libraries, the parent institutions received a return on investment of between  0.64:1 and 15.54:1 in research grant income.
  2. In two North American universities, regression analysis using 10 years of data shows that an increase in the library budget is correlated with an increase in grant funding.
  3. Faculty survey respondents cite averages of 7.5 to 41.2 books or articles in each grant proposal they write.
  4. At least three-fourths of survey respondents state it is “important,” “very important,” or “essential” to cite references to journal articles or books in their grant proposals.
  5. Most respondents access at least half of the articles and books they cite in grant proposals, reports, and publications from their institutional library e-resource collections.
  6. Survey respondents report that they spend at least 3.5 hours per week finding and accessing articles, and at least 9.8 hours reading articles.

(Executive Summary)

Site Admin

© 2010-2012 Association of College & Research Libraries, a division of the American Library Association

Suffusion theme by Sayontan Sinha