Disruptive Educational Models and Open Education

Eating Your Own Dog Food

One of the most memorable experiences I had as a library student was becoming a patron of my own library. As on online library school student* I usually worked either in my office at pre-approved times, or at home. However, depending on the assignment, sometimes I worked out at the reference area public access computers. It nearly drove me mad, for a very simple reason – this was in the day before optical mouse devices, and the trackballs on our mice were incredibly sticky and jerky, despite regular cleaning routines. It was so bad I wondered how students could stand to work on our workstations, and how it made them feel about the library in general, since there is nothing like a solid hour or so of constantly repeated, albeit small, irritations to make a person develop indelible negative feelings towards a particular environment.

I’ve heard the same thing from colleagues that have started graduate programs here at my university; they are shocked at how hard it can be to be a student in the library, even with insider knowledge, and it can be demoralizing (and galvanizing) to watch classmates and even instructors dismiss library services and resources with “too confusing” or “learning curve too steep” as they ruthlessly practice least-effort satisficing for their information needs.

In information technology circles, the concept of having to use your own platforms/services is known as “eating your own dog food” or “dogfooding.” While there are pitfalls to relying too heavily on it as an assessment tool (we all have insider knowledge about libraries, software, and resources that can smooth the process for us), it is an eye-opening exercise, especially to listen to our users be brutally frank about what we offer — or don’t.

DIY Universities and Open Education

I am suggesting something related but complementary to dogfooding — sampling the models and platforms of a burgeoning movement that has the potential to be a disruptive force in higher education. DIY U and the coming transformation of education are all the rage (pun intended) these days, as prestigious universities and professors, Edupunks, loose collaboratives, and start-ups participate in collaborative free online offerings through various platforms and with different aims: CourseraKhan AcademyP2PUMIT OpenCourseWareUdacityNYU Open Education, and many more. This is a call to action for us as librarians. Instead of endlessly debating what this might mean, or where it might be going, and this movement’s possible effect on academic libraries, I suggest actually signing up for a course and experiencing it first-hand.

For library technologists facing the brave new world of higher education in the 21st century, there are three major advantages to taking a class in one of the new experimental DIY universities. We get to experience new platforms, delivery mechanisms, and modes of teaching, some of which may be applicable to the work of the academic library. In addition, many of the courses offered are technical courses that are directly applicable to our daily work. Thirdly, it allows us as academic participants to personally assess the often intemperate and hyperbolic language on both sides of the debate: “can’t possibly be as good as institutional campus-based face-to-face EVER” versus “This changes everything, FOREVER.” How many faculty on your campuses do you think have actually taken an online class, especially in one of these open educational initiatives? This is an opportunity to become an informed voice in any local campus debates and conversations. These conversations and debates will involve our core services, whether faculty and administrators realize it or  not.

It will also encourage some future-oriented thinking about where libraries could fit into this changing educational landscape. One of the more interesting possible effects in these collaborative,  open-to-all ventures is the necessity of using free or open access high quality resources. Where will that put the library? What does that mean for instructional resources hidden behind a particular institution’s authentication wall? Academic libraries and services have been tied to a particular institution — what happens when those affiliations blur and change extremely rapidly? There are all sorts of implications for faculty, students, libraries, vendors, and open access/open educational resources platforms. As a thought exercise, take a look at these seven predictions for the future of technology-enabled universities from JISC’s Head of Innovation, Sarah Porter. Which ones DON’T involve libraries? As a profession, let’s get out on the bleeding edge and investigate the developing models.

I just signed up for “Model Thinking” through Coursera. Taught by Professor Scott E. Page from the Center for the Study of Complex Systems at the University of Michigan, the course will cover modeling information to make sense of trends, social movements, behaviors, because “evidence shows that people who think with models consistently outperform those who don’t. And, moreover people who think with lots of models outperform people who use only one.” That sounds applicable to making decisions about e-books, collection development, workflow redesign, and changing models of higher education, et cetera.

Some Suggestions:

  • Coursera offers clusters of courses in Society, Networks, and Information (Model Thinking, Gamification, Social Networking Analysis, among others) and Computer Science (Algorithims, Compilers, Game Theory, etc.). If you have a music library or handle streaming media in your library, what about Listening to World Music? If you are curious about humanities subjects that have depended on traditional library materials in the past, try A History of the World since 1300 or Greek and Roman Mythology.
  • Udacity offers Building a Search Engine, Design of Computer Programs, and Programming a Robotic Car (automate a bookmobile?).
  • Set up your own peer class with P2PU, or take Become a Citizen Scientist, Curating Content, or Programming with the Twitter API.
  • If you are in the New York City area and can attend an in-person workshop, General Assembly offers Storytelling Skills, Programming Fundamentals for Non-Programmers, and Dodging the Dangers of Copyright Law (taught by participants in Yale Law School’s Information Society Project) as part of a menu of  tech and tech-business related workshops. These have fees ranging from $15 to $30.
  • Before I take my Model Thinking class, I’m planning to brush up my algebra at Khan Academy.
  • Try the archived lectures at Harvard’s “Building Mobile Applications“, hosted in their institutional repository.
  • Health Sciences Librarian? What about Information Technology in the Health Care System of the Future from MIT OpenCourseWare?

 

* Full disclosure: I am a proud graduate of University of Illinois’ LEEP (5.0) MSLIS program, and I also have another master’s degree done the old fashioned way, and I am an enthusiastic supporter of online education done correctly.

Glimpses into user behavior

 

Heat map of clicks on the library home page
Heat map of clicks on the library home page

Between static analytics and a usability lab

Would you like an even more intimate glimpse into what users are actually doing on your site, instead of what you (or the library web committee)  think they are doing? There are several easy-to-use web-based analytics services like ClickTale , userflyLoop11Crazy Egg, Inspectlet, or Optimalworkshop. These online usability services offer various ways to track what users are doing as they actually navigate your pages — all without setting up a usability lab, recruiting participants, or introducing the artificiality and anxiety of an observed user session. ClickTale and userfly record user actions that you can view later as a video; most services offer heatmaps of where users actually click on your site; some offer “eye tracking” maps based on mouse movement.

  • Most services allow you to sign up for one free account for a limited amount of data or time.
  • Most allow you to specify which pages or sections of your site that you want to test at a time.
  • Many have monthly pricing plans that would allow for snapshots of user activity in various months of the year without having to pay for an entire year’s service.

We’re testing Inspectlet at the moment. I like it because the free account offers the two services I’m most interested in: periodic video captures of the designated site and heat maps of actual clicks. The code is a snippet added to the web pages of interest. The screen captures are fascinating — watch below as an off-campus user searches the library home page for the correct place to do an author search in the library catalog. I view it as a bit of a cautionary illustration about providing a lot of options. Follow the yellow “spotlight” to track the user’s mouse movements. As a contrast, I watched video after video of clearly experienced users taking less than two seconds to hit the “Ebsco Academic Search” link. Be prepared; watching a series of videos of unassisted users can dismantle your or your web committee’s cherished notions about how users navigate your site.

Inspectlet video thumbnail

This is a Jing video of a screen capture — the actual screen captures are much sharper, and I have zoomed out for illustrative purposes. The free Inspectlet account does not support downloads of capture videos, but Rachit Gupta, the founder, wrote me that in the coming few weeks, Inspectlet is releasing a feature to allow downloads for paid accounts. Paid accounts also have access to real time analytics, so libraries would be able to get a montage of what’s happening in the lobby as it is happening. Imagine being able to walk out and announce a “pop-up library workshop” on using the library catalog effectively after seeing the twentieth person fumble through the OPAC.

Another thing I like about Inspectlet is the ability to anonymize the IP addresses in the individual screen captures to protect an individual patron’s privacy.

The chart below compares the features of a few of the most widely used web-based analytics tools.

 

Vendor Video Captures Heat Maps Mouse & Click Tracking Real Time mode Other Privacy Policy Pricing plan
ClickTale

Scroll maps, form analytics, conversion funnels, campaigns Privacy Policy Basic $99/month; limited free plan; month to month pricing; higher
education discounts available (call)
Crazy Egg  

  Scroll maps, click overlays, confetti overlay Privacy Policy Basic $9/month (annual)
Inspectlet

Scroll  maps, Custom API, anonymized IP addresses Privacy Policy Starter $7.99/month; limited free account.Can cancel subscription at any time.
mouseflow

Movement heatmaps, link analytics Privacy Policy Small: aprox $13 US/month; free plan.Can cancel subscription at any time.
seevolution

Scroll maps, visual tool set for real  time Privacy Policy Light: $29/month. Free plan, but very limited details.
userfly

Terms (with a brief privacy explanation) Basic $10/month; free 10 captures a monthCan cancel subscription at any time.

If you are using one of these services, or a similar service, what have you learned about your users?

Testing new designs or alternative designs – widely used web-based usability tools

After you’ve watched your users and determined where there are problems or where you would like to try an alternative design,  these services offer easy ways to test new designs and gather feedback from users without setting up a local usability lab.

 

Loop11 Create test scenarios and analyze results (see demo) Privacy Policy First project free; $350 per project
Optimalworkshop  Card sorting, Tree Testing, Click Testing Privacy Policy Free plan small project; $109 for each separate plan; 50% discount for education providers
OpenHallway Create test scenarios and analyze results Terms of Service Basic: $49/month; limited free account, Can cancel subscription at any time.
Usabilla Create test scenarios and analyze results; mobile UX testing Terms of Service Starter: $19/month. Can cancel subscription at any time.

 

Action Analytics

What is Action Analytics?

If you say “analytics” to most technology-savvy librarians, they think of Google Analytics or similar web analytics services. Many libraries are using such sophisticated data collection and analyses to improve the user experience on library-controlled sites.  But the standard library analytics are retrospective: what have users done in the past? Have we designed our web platforms and pages successfully, and where do we need to change them?

Technology is enabling a different kind of future-oriented analytics. Action Analytics is evidence-based, combines data sets from different silos, and uses actions, performance, and data from the past to provide recommendations and actionable intelligence meant to influence future actions at both the institutional and the individual level. We’re familiar with these services in library-like contexts such as Amazon’s “customers who bought this item also bought” book recommendations and Netflix’s “other movies you might enjoy”.

BookSeer β by Apt

Action Analytics in the Academic Library Landscape

It was a presentation by Mark David Milliron at Educause 2011 on “Analytics Today: Getting Smarter About Emerging Technology, Diverse Students, and the Completion Challenge” that made me think about the possibilities of the interventionist aspect of analytics for libraries.  He described the complex dependencies between inter-generational poverty transmission, education as a disrupter, drop-out rates for first generation college students, and other factors such international competition and the job market.  Then he moved on to the role of sophisticated analytics and data platforms and spoke about how it can help individual students succeed by using technology to deliver the right resource at the right time to the right student.  Where do these sorts of analytics fit into the academic library landscape?

If your library is like my library, the pressure to prove your value to strategic campus initiatives such student success and retention is increasing. But assessing services with most analytics is past-oriented; how do we add the kind of library analytics that provide a useful intervention or recommendation? These analytics could be designed to help an individual student choose a database, or trigger a recommendation to dive deeper into reference services like chat reference or individual appointments. We need to design platforms and technology that can integrate data from various campus sources, do some predictive modeling, and deliver a timely text message to an English 101 student that recommends using these databases for the first writing assignment, or suggests an individual research appointment with the appropriate subject specialist (and a link to the appointment scheduler) to every honors students a month into their thesis year.

Ethyl Blum, librarian

Privacy Implications

But should we? Are these sorts of interventions creepy and stalker-ish?* Would this be seen as an invasion of privacy? Does the use of data in this way collide with the profession’s ethical obligation and historical commitment to keep individual patron’s reading, browsing, or viewing habits private?

Every librarian I’ve discussed this with felt the same unease. I’m left with a series of questions: Have technology and online data gathering changed the context and meaning of privacy in such fundamental ways that we need to take a long hard look at our assumptions, especially in the academic environment? (Short answer — yes.)  Are there ways to manage opt-in and opt-out preferences for these sorts of services so these services are only offered to those who want them? And does that miss the point? Aren’t we trying to influence the students who are unaware of library services and how the library could help them succeed?

Furthermore, are we modeling our ideas of “creepiness” and our adamant rejection of any “intervention” on the face-to-face model of the past that involved a feeling of personal surveillance and possible social judgment by live flesh persons?  The phone app Mobilyze helps those with clinical depression avoid known triggers by suggesting preventative measures. The software is highly personalized and combines all kinds of data collected by the phone with self-reported mood diaries. Researcher Colin Depp observes that participants felt that the impersonal advice delivered via technology was easier to act on than “say, getting advice from their mother.”**

While I am not suggesting in any way that libraries move away from face-to-face, personalized encounters at public service desks, is there room for another model for delivering assistance? A model that some students might find less intrusive, less invasive, and more effective — precisely because it is technological and impersonal? And given the struggle that some students have to succeed in school, and the staggering debt that most of them incur, where exactly are our moral imperatives in delivering academic services in an increasingly personalized, technology-infused, data-dependent environment?

Increasingly, health services, commercial entities, and technologies such as browsers and social networking environments that are deeply embedded in most people’s lives, use these sorts of action analytics to allow the remote monitoring of our aging parents, sell us things, and match us with potential dates. Some of these uses are for the benefit of the user; some are for the benefit of the data gatherer. The moment from the Milliron presentation that really stayed with me was the poignant question that a student in a focus group asked him: “Can you use information about me…to help me?”

Can we? What do you think?

* For a recent article on academic libraries and Facebook that addresses some of these issues, see Nancy Kim Phillips, Academic Library Use of Facebook: Building Relationships with Students, The Journal of Academic Librarianship, Volume 37, Issue 6, December 2011, Pages 512-522, ISSN 0099-1333, 10.1016/j.acalib.2011.07.008. See also a very recent New York Times article on use of analytics by companies which discusses the creepiness factor.