Glimpses into user behavior

 

Heat map of clicks on the library home page
Heat map of clicks on the library home page

Between static analytics and a usability lab

Would you like an even more intimate glimpse into what users are actually doing on your site, instead of what you (or the library web committee)  think they are doing? There are several easy-to-use web-based analytics services like ClickTale , userflyLoop11Crazy Egg, Inspectlet, or Optimalworkshop. These online usability services offer various ways to track what users are doing as they actually navigate your pages — all without setting up a usability lab, recruiting participants, or introducing the artificiality and anxiety of an observed user session. ClickTale and userfly record user actions that you can view later as a video; most services offer heatmaps of where users actually click on your site; some offer “eye tracking” maps based on mouse movement.

  • Most services allow you to sign up for one free account for a limited amount of data or time.
  • Most allow you to specify which pages or sections of your site that you want to test at a time.
  • Many have monthly pricing plans that would allow for snapshots of user activity in various months of the year without having to pay for an entire year’s service.

We’re testing Inspectlet at the moment. I like it because the free account offers the two services I’m most interested in: periodic video captures of the designated site and heat maps of actual clicks. The code is a snippet added to the web pages of interest. The screen captures are fascinating — watch below as an off-campus user searches the library home page for the correct place to do an author search in the library catalog. I view it as a bit of a cautionary illustration about providing a lot of options. Follow the yellow “spotlight” to track the user’s mouse movements. As a contrast, I watched video after video of clearly experienced users taking less than two seconds to hit the “Ebsco Academic Search” link. Be prepared; watching a series of videos of unassisted users can dismantle your or your web committee’s cherished notions about how users navigate your site.

Inspectlet video thumbnail

This is a Jing video of a screen capture — the actual screen captures are much sharper, and I have zoomed out for illustrative purposes. The free Inspectlet account does not support downloads of capture videos, but Rachit Gupta, the founder, wrote me that in the coming few weeks, Inspectlet is releasing a feature to allow downloads for paid accounts. Paid accounts also have access to real time analytics, so libraries would be able to get a montage of what’s happening in the lobby as it is happening. Imagine being able to walk out and announce a “pop-up library workshop” on using the library catalog effectively after seeing the twentieth person fumble through the OPAC.

Another thing I like about Inspectlet is the ability to anonymize the IP addresses in the individual screen captures to protect an individual patron’s privacy.

The chart below compares the features of a few of the most widely used web-based analytics tools.

 

Vendor Video Captures Heat Maps Mouse & Click Tracking Real Time mode Other Privacy Policy Pricing plan
ClickTale

Scroll maps, form analytics, conversion funnels, campaigns Privacy Policy Basic $99/month; limited free plan; month to month pricing; higher
education discounts available (call)
Crazy Egg  

  Scroll maps, click overlays, confetti overlay Privacy Policy Basic $9/month (annual)
Inspectlet

Scroll  maps, Custom API, anonymized IP addresses Privacy Policy Starter $7.99/month; limited free account.Can cancel subscription at any time.
mouseflow

Movement heatmaps, link analytics Privacy Policy Small: aprox $13 US/month; free plan.Can cancel subscription at any time.
seevolution

Scroll maps, visual tool set for real  time Privacy Policy Light: $29/month. Free plan, but very limited details.
userfly

Terms (with a brief privacy explanation) Basic $10/month; free 10 captures a monthCan cancel subscription at any time.

If you are using one of these services, or a similar service, what have you learned about your users?

Testing new designs or alternative designs – widely used web-based usability tools

After you’ve watched your users and determined where there are problems or where you would like to try an alternative design,  these services offer easy ways to test new designs and gather feedback from users without setting up a local usability lab.

 

Loop11 Create test scenarios and analyze results (see demo) Privacy Policy First project free; $350 per project
Optimalworkshop  Card sorting, Tree Testing, Click Testing Privacy Policy Free plan small project; $109 for each separate plan; 50% discount for education providers
OpenHallway Create test scenarios and analyze results Terms of Service Basic: $49/month; limited free account, Can cancel subscription at any time.
Usabilla Create test scenarios and analyze results; mobile UX testing Terms of Service Starter: $19/month. Can cancel subscription at any time.

 

The End of Academic Library Circulation?

What Library Circulation Data Shows

Unless current patterns change, by 2020 university libraries will no longer have circulation desks. This claim may seem hyperbolic if you’ve been observing your library, or even if you’ve been glancing over ACRL or National Center for Education Statistics data. If you have been looking at the data, you might be familiar with a pattern that looks like this:

total circulationThis chart shows total circulation for academic libraries, and while there’s a decline it certainly doesn’t look like it will hit zero anytime soon, definitely not in just 8 years. But there is a problem with this data and this perspective on library statistics.  When we talk about “total circulation” we’re talking about a property of the library, we’re not really thinking about users.

Here’s another set of data that you need to look at to really understand circulation:
fall enrollmentsAcademic enrollment has been rising rapidly.  This means more students, which in turns means greater circulation.  So if total circulation has been dropping despite an increase in users then something else must be going on.  So rather than asking the question “How many items does my library circulate?” we need to alter that to “How many items does the average student checkout?”

Here is that data:

circulation per student

This chart shows the upper/lower quartiles and median for circulation per FTE student.  As you can see this data shows a much more dramatic drop in the circulation of library materials. Rising student populations hide this fact.

xkcd
[source: http://xkcd.com/605/]

But 2020? Can I be serious?  The simple linear regression model in the charts is probably a good predictor of 2012, but not necessarily 2020. Hitting zero without flattening out seems pretty unlikely. However, it is worth noting the circulation per user in the lower quartile for less than 4 year colleges reached 1.1 in 2010. If you’re averaging around 1 item per user, every user that takes out 2 items means there’s another who has checked out 0.

What’s Happening Here?

Rather than waste too much time trying to predict a future we’ll live in in less than a decade, let’s explore the more interesting question: “What’s happening here?”

By far the number one hypothesis I get when I show people this data is “Clearly this is just because of the rise of e-journals and e-books”. This hypothesis is reasonable: What has happened is simply that users have switched from print to electronic. This data represents a shift in media, nothing more.

But there are 2 very large problems with this hypothesis.

First, print journal circulation is not universal among academic libraries. In the cases where there is no print journal circulation the effect of e-journals would not be present in circulation data. However, I don’t have information to point out exactly how many academic libraries did circulate print journals. Maybe the effect of e-journals on just the libraries that do circulate serials could effect the data for everyone. The data we have already shown resolves this issue. Libraries that did circulate serials would have higher circulation per user than those that did not. By showing different quartiles we can address this discrepancy in the data between libraries that did and did not circulate journals. If you look at the data you’ll see that indeed the upper quartile does seem to have a higher rate of decline, but not enough to validate this hypothesis. The median and lower quartiles also experience this shift, so something else must be at work.

Second, e-books were not largely adopted until the mid 2000s, yet the decline preceding 2000 is at least as steep as after. If you look at the chart below you’ll notice that ebook acquisition rates did not exceed print until 2010:

ebooks vs printEbooks, of course, do have an effect on usage, but they’re not the primary factor in this change.

So clearly we must reject the hypothesis that this is merely a media shift. Certainly the shift from print to electronic has had some effect, but it is not the sole cause. If it’s not a shift in media, the most reasonable explanation is that it’s a shift in user behavior.  Students are simply not using books (in any format) as much as they used to.

What is Causing this Shift in User Behavior?

The next question is what is the cause of this shift.

I think the most simple answer is the web. 1996 is the first data point showing a drop in circulation. Of course the web was quite small then, but AOL and Yahoo! were already around, and the Internet Archive had been founded.  If you think back to a pre-web time, pretty much anything you needed to know more about required a trip to the library and checking out a book.

The most important thing to take away is that, regardless of cause, user behavior has changed and by all data points is still changing.  In the end, the greatest question is how will academic libraries adapt?  It is clear that the answer is not as simple as a transition to a new media. To survive, librarians must find the answer before we have enough data to prove these predictions.

If you enjoyed exploring this data please check out Library Data and follow @librarydata on twitter.

Data Source:

About our guest author: Will Kurt is a software engineer at Articulate Global, pursuing his masters in computer science at the University of Nevada, Reno and is a former librarian. He holds an MLIS from Simmons College and has worked in various roles in public, private and special libraries at organizations such as: MIT, BBN Technologies and the University of Nevada, Reno. He has written and presented on a range of topics including: play, user interfaces, functional programming and data
visualization.