Author: Lauren Magnuson

I’ve always gravitated toward library jobs in library systems and technology, but I recently took on a new position as head of a tech services department in a smaller academic library.  Some of my colleagues expressed surprised that I’m moving out of a traditional library IT or systems role, but my former position was as a systems librarian within a technical services department, and for the past few years, a significant amount of my time recently has involved developing collection and metadata-related system integrations for acquisitions and cataloging. A few trends have made me think that I’m not alone in branching out and applying systems skills to diverse functional areas of the library.  It has become relatively commonplace for the work of technology innovation to occur, at least in part, outside of traditional library IT departments; for example, reference and instruction librarians playing a tightly integrated role in the optimization of discovery interfaces, tech services staff using Python and linked data technologies to clean up and enhance metadata, and instruction librarians and access services staff creating and managing high-tech MakerSpaces.

More personnel across the library are embracing and developing high tech skills traditionally housed in library systems or IT departments.  The following are six general trends I’ve observed that are influencing the spread of technology development outside of traditional library IT.

Increasingly high technical skills are required for most library areas

• Job advertisements for almost every functional area of the library emphasize advanced technical knowledge (beyond typical office application knowledge), especially with regard to ILS systems management.  In a 2016 study of library job advertisements, the authors found a wide range of job titles that require knowledge and skills in information technology, including Metadata Librarian, Digital Archivist, Information Literacy Librarian, and Research Data Librarian (Shahbazi, Fahimnia, & Khoshemehr, 2016).   Scholarly communications, data services, e-resource management, reference, and other library staff positions may all be positioned outside of traditional library IT, but are all deeply involved in the utilization and development of library technologies.

Optimization of cloud-based systems can be distributed

• With cloud-based application hosting, managing physical servers and backups may become less burdensome, but the need for knowledgeable personnel to configure and optimize often complex cloud-based systems is as essential as it has always been.  Scholarly communications, e-resource, and access services library staff may play highly integrated roles in the development and optimizing of library systems.  Beyond acting as consultants for the management of these systems, knowledge in scripting and interoperability mechanisms enables staff outside of library IT to holistically contribute to the development of cloud-based library applications.

More opportunities to build integrations

• New library services platforms often enable a number of integrations with third party systems via APIs and other web services.¹ Many of these integrations require a deep knowledge of workflows and data structures between multiple systems, so including involvement from multiple functional areas is usually required. The combination of knowledge of a functional area with knowledge of how the library system works can result in some seriously powerful and useful applications.

The increasing importance of data wrangling

• Sources of metadata are increasingly varied, requiring data wrangling, work that is often made more efficient by developing coding or scripting methods to automate routine tasks.  Metadata specialists are often experts in developing macros in OCLC Connexion (for example), and increasingly require access to a computing environment that enables writing, testing, and using code to further automate metadata cross-walking and cleanup, including use of Python, OpenRefine, and other tools for dealing with enormous amounts of messy data.

Customization of discovery and library e-content

• Discovery platforms are complex and often highly customizable. Many reference and instruction librarians have a robust understanding of user behavior and information literacy goals that are essential for development of usable interfaces, as well as skills in user experience (UX) testing and interface design.  Reference and instruction librarians are often experts in course management systems and LibGuides, and know good tricks and hacks for optimizing digital learning content.  Library collection development, scholarly communications and technical services librarians deeply understand content and how to make it findable, and increasingly play pivotal roles in configuring harvesting and transformation of metadata into discovery systems.

Systems beyond the ILS

• Libraries are engaging with a much wider variety of technologies than just an ILS – libraries support institutional repository and digital library software, data management software, authority systems such as VIAF, open publishing, etc. While working with these systems does not necessarily require a background or emphasis on systems administration, it is definitely helpful to have an understanding of the architecture of such applications and how applications might interact with each other.
• Systems knowledge is applicable to more than just technology. Thinking like a programmer can often be useful when performing workflow analysis and optimization, as well as problem-solving even in non-technical areas.

Are “true” library systems administrators still needed? (Yes, obviously)

When researching for this post, I came across an amusing article by Roy Tennant from back in 2011 titled “If You are a Library SysAdmin, you are TOAST”.  The article presents a (seemingly not satirical?) argument that movement to cloud-based systems in libraries will make library system administrators obsolete:

When I, as just a moderately savvy librarian, can learn maybe five to ten very specific steps and be able to deploy any application I would likely want to deploy, why do I need to talk to my system administrator ever again?

Obviously, six years after this article was written, with many libraries firmly embedded in the cloud with a variety of library applications, the role of the system administrator in library is not at all diminished.  While work involving physical servers and backups may be less common for many applications, system administrators and those with IT skills in libraries are still in huge demand to be on hand to evaluate, optimize, and provide integrations for cloud-based library systems.

I think it’s safe to say that the more people in any organization with technical knowledge, the better.  Managing decentralized technology projects, however, does require leadership and coordination.  When learning to develop applications, coding is often much more fun than worrying about server administration and security – but of course, someone has to be concerned about security and help those who may just be learning about technology adopt secure development practices.  Library technology projects don’t have to come out of library IT departments, but leadership from library IT departments should be open and supportive of library technology initiatives coming out of non-library IT areas, while facilitating secure practices.  Coordination on the part of library IT is also essential to avoid duplication of effort and ensure that projects being developed are sustainable and supported by the technology environment of the larger organization.  Encouraging the open exchange of technology-related ideas across the library prevents tech savvy staff feeling they need to hide their pet projects lest they get ‘in trouble’ with a restrictive library IT department.

In my view, there’s simply too much technology change happening in library to keep all technology development centralized in a single unit within the library.  Adding tech-savvy positions within non-technology departments is not a bad strategy – it can help support innovation out of library departments that haven’t traditionally been expected to drive technological change.  However, continually raising expectations with regard to technical knowledge can be stressful, so ensuring that strong support for professional development is in place is also important.  In my own new position, I’m excited to channel my knowledge of APIs and interest in data visualization technologies into creating some cool collection management and assessment tools, and I’m not at all concerned that I won’t have an opportunity to apply my technical knowledge in the rapidly changing landscape of library technical services and collection development.  Working outside of library IT means that I need to communicate closely with the head of library IT about the projects I’m working on, and also be sure to closely follow other technology-related projects across the library and be proactive about offering my skills where they might be helpful.  It also means that I need to work to support the technical expertise of staff in my department, particularly as related to library system management in acquisitions and cataloging.    No matter your role in the library, there’s plenty of technology-related work to go around.

Learning Tools Interoperability, or LTI, is an open standard maintained by the IMS Global Learning Consortium used to build external tools or plugins for Learning Management Systems (LMS).  A common use case of an LTI is to build an application that can be accessed from within the LMS to perform searches and import resources into a course.  For example, the Wikipedia LTI application enables instructors to search Wikipedia and embed links to articles directly into their courses.  Academic libraries frequently struggle to integrate library resources in learning management systems, so LTI is an obvious standard to embrace as a potential way to make library resources more accessible.  However, when I began researching how I could begin creating an LTI app, I found it very difficult to find examples of existing app code and resources to get started.  You can’t just create any old web application and have that be ‘consumable’ by a learning management system in an LTI-compliant way.  In this post, I’ll outline some of the resources I found useful to get started building your own LTI app.

LTI General Architecture

These are the basic components of an LTI application:

• The LTI Tool Provider (TP):  This is your application.  The tool provider is the resource the user sees when they access your application from within the learning management system.  The Wikipedia LTI app linked above is an example of a tool provider.
• The LTI Tool Consumer (TC): This is the learning management system (e.g., Blackboard, Moodle, Canvas) from which the user accesses your tool provider application.
• The LTI Launch:  When a user accesses your tool provider from the tool consumer, this is called “launching” the LTI application.  Parameters are passed from the tool consumer to your tool provider, including authorization parameters that ensure the user is permitted to access your application, as well as information about the user’s identity, roles within the tool consumer, and the type of request the user is sending (e.g., a “content item message” is sent to your tool to indicate the user is expecting to import a link back to the tool consumer).
• OAuth:  LTI applications use OAuth signatures for validating messages between the Tool Consumer and the Tool provider.  LTI applications require that the Tool Consumer and the Tool Consumer have each configured a shared key and secret, which is used to build an OAuth “Access Token” to enable communication between the two systems.¹

An additional tip for developing LTI apps:  Sign up for a free instructor account for the Canvas learning management system. Canvas accounts hosted on the Instructure website enable you to add a custom LTI tool to your course (once it’s hosted on a web server, of course) and also enables you to quickly experiment with some existing LTI applications (such as the Khan Academy and Merlot LTI apps) to explore possible functionality you might want to include in your application.  This way, you can see what an instructor or student would see when they interact with your tool provider through an LMS.

Building your first “Hello, World” LTI app (with some help from Harvard)

When I first started looking into LTI, I found it really difficult to find a full (but basic) LTI application to get an overall picture of how LTI apps work – there’s lots of LTI class libraries out there, but I wanted an example of how all the pieces of an LTI app fit together. After some fruitless Googling and GitHub searches, I finally stumbled upon this Harvard LTI workshop on LTI apps that really helped me understand how LTI applications work.  The repository includes a full working LTI application that you can simply “plug in” some basic values to create a fully working LTI application, complete with OAuth authentication.

First, be sure to look at the included presentation in the repository, which is a rare example of a set of presentation slides that is 100% understandable out of context, to get a general introduction to the LTI standard and what it attempts to achieve.  You’ll also want to read through the step-by-step LTI blog tutorial that will get you set up with your first “Hello,World!” LTI application, complete with valid OAuth-signed requests.²

I found it especially useful that the Harvard LTI workshop repository includes a pseudo tool consumer (which mimics how the LMS would interact with your tool) that you can use during development on localhost.  Once you follow the steps of the tutorial to build your basic “Hello, World!” single-page LTI application, you can plug the local URL of that into the tool consumer page and check out how the parameters are passed from the tool consumer to the tool provider.   You can also examine the built-in basic LTI php class library that is included, as well as the basic OAuth functionality to see how the OAuth Access Token is constructed.

Use Case:  A WorldCat Discovery API Search and Retrieval Tool for LMS

My particular use case for exploring LTI involves building a search box that would enable a faculty user to add a link to a resource from the WorldCat Discovery system.  If your library subscribed to FirstSearch or you are a WorldShare Management System (WMS) customer you now likely have access to WorldCat Discovery; but the framework I’m using to build my app would work for any Discovery layer with an API (e.g., Summon, Primo, etc.).

Searching and retrieving via LTI is straightforward.  First, using the Harvard LTI workshop LTI application, I created a /lib directory to host the WorldCat Discovery PHP library published by OCLC cloned from GitHub.  I installed the library using Composer as described in the GitHub repository readme instructions.   I created a very simple search form and response page that enables a user to enter a query and then retrieve results from the WorldCat Discovery API based on that query. Then, I set up my “tool.php” application to display the search form and POST the query to the the simple response page:

tool.php:

<?php
error_reporting(E_ALL & ~E_NOTICE);
ini_set("display_errors", 1);
require_once 'ims-blti/blti.php';
$lti = new BLTI("secret", false, false);

session_start();
header('Content-Type: text/html;charset=utf-8');
?>

<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8" />
    <title>Building Tools with the Learning Tools Operability Specification</title>
  </head
  <body>
  <?php 
    if ($lti->valid) {
  ?>
    <h2>Search WorldCat Discovery</h2>
      <form action="results.php" method="post" encType="application/x-www-form-urlencoded">
        Search: <input type="text" name="query" id="query" />
         <?php
    foreach($_POST as $key => $value) {
      echo "<input type=\"hidden\" name=\"" .$key .  "\" value=\"" . $value . "\" />\n";
    }
  ?>
      <input type="submit" name="submit" value="Submit" />
      </form>
    <pre>
    </pre>
    <?php
      } else {
    ?>
      <h2>This was not a valid LTI launch</h2>
      <p>Error message: <?= $lti->message ?></p>
    <?php
      }
    ?>
    </body>
</html>

results.php:

<?php
require_once('../lib/worldcat-discovery-php/vendor/autoload.php');

   use OCLC\Auth\WSKey;
   use OCLC\Auth\AccessToken;
   use WorldCat\Discovery\Bib;

$key = 'somekey';
$secret = 'somesecret';
$options = array('services' => array('WorldCatDiscoveryAPI', 'refresh_token'));
$wskey = new WSKey($key, $secret, $options);
$accessToken = $wskey->getAccessTokenWithClientCredentials('123', '123');

$query = $_POST["query"];
$options = array(
  'useFRBRGrouping' => 'true',
  'sortBy' => 'library_plus_relevance',
  'itemsPerPage' => 25,
  );
$bib = Bib::Search($query, $accessToken, $options);

if (is_a($bib, 'WorldCat\Discovery\Error')) {
   echo $bib->getErrorCode();
   echo $bib->getErrorMessage();
} else {
    foreach ($bib->getSearchResults() as $result){
      echo '<li><a href="'. $result . '">' . $result->getName()->getValue() .' (';
      echo ($result->getDatePublished() ?  '' . $result->getDatePublished()->getValue()  : '') . ')</a></li>';
   }
}

?>

The application I’ve created so far is mostly a proof of concept, and I have a few essential tasks to finish the application – first, I need to re-write the URLs to point to a specific WorldCat Discovery instance (pointing to generic WorldCat.org isn’t helpful when a user is wanting to embed the resources of a specific library, to enable full-text access links and ); second, my app needs to enable the user to return these links to the LMS so that students / course participants can click on them.

For the second point, there is an LTI specification called the “content-item-message” that indicates that the type of interaction requested from the tool is the return of a link to the LMS.  The LMS must include this input parameter in the POST request to the tool.  The LMS “knows” to send this parameter when the tool is initially installed in the LMS.

<input type="hidden" name="lti_message_type" value="ContentItemSelectionRequest" />

The POST request to the tool must also indicate the return URL (e.g., the URL back to the LMS) where the link should be sent (the LMS should generate this input parameter for you; your tool just needs to identify this parameter and include it in the POST request to return the link to the LMS):

<input type="hidden" name="content_item_return_url" value="http://www.tc.com/item-return" />

The Tool provider must then render the link to be imported with some description of the content in JSON, for example:

{
  "@context" : "http://purl.imsglobal.org/ctx/lti/v1/ContentItem", 
  "@graph" : [ 
    { "@type" : "LtiLinkItem",
      "url" : "https://someinstitution.worldcat.org/oclc/709669613",
      "mediaType" : "text/html",
      "title" : "Global Warming: Hype or Hazard?"
    }
  ]
}

See the Content Item Message documentation for more details on returning JSON suitable for consumption by the LMS.

Learn, and do, more with LTI

You may find the basic LTI class script included in the Harvard LTI tutorial are insufficient for your use case – the code is a bit aged and the LTI specification has moved on.

A more robust LTI tool provider PHP library than the basic one included in the Harvard tutorial has been made available by IMS Global on GitHub.  You can also find a more complex complete sample app called “Rating” that is a great example of more complex kinds of interactions with an LTI app, including how you might build a server-side data store and recall that data through the LTI app, and how you might handle the assignment of grades or scores through an LTI app.

To learn more, the Canvas Learning Management system has an excellent open course on LTI development that you can enroll yourself in with a free Canvas account.  Once enrolled in the course, you can launch your own locally developed LTI app within the course to check how parameters and data are exchanged between the LMS and the tool.

Python is a great programming language to know if you work in a library: it’s (relatively) easy to learn, its syntax is fairly clear and intuitive, and it has great, robust libraries for doing routine library tasks like hacking MARC records and working with delimited data, CSV files, JSON and XML. ¹  In this post, I’ll describe a couple of projects I’ve worked on recently that have enabled me to Do Library Stuff Faster using Python.  For reference, both of these scripts were written with Python 2.7 ² in mind, but could easily be adapted for other versions of Python.

Library Holdings Lookup with Beautiful Soup

Here’s a very common library dilemma:  A generous and well-meaning patron, faculty member, or friend of the library has a large personal collection of books or other materials that they would like to bequeath to your library.  They have carefully created a spreadsheet (or word document, or hand-written index) of all of the titles and authors (and maybe dates and ISBNs) in their library and want to know if you want the items.

Many libraries (for very good reason) have policies to just say “no” to these kinds of gifts.  Well-meaning library gift givers don’t always realize that it’s an enormous amount of work for a library to evaluate materials and decide whether or not they can be added to the library’s collection. Beyond relevance to their users and condition of the items, libraries don’t want to accept gifts of duplicate copies of titles they already have in their collection due to limited shelf space.

It’s that final point – how to avoid adding duplicate titles to the collection – that led me to develop a very simple (and very hacky) script to enable me to take a list of titles and authors and do a very simple lookup to see if, at minimum, we have those same titles already in the collection.  Our ILS (Innovative Interface’s Millennium system) does not have a way to feed in a bunch of titles and generate a report of title matches – and I would venture to say that kind of functionality is probably not available in most library systems.  Normally when presented with a dilemma of having to check to see if the library already has a set of titles, we’d sit down an unfortunate student worker and have them manually work through the list – copying and pasting titles into the library catalog and noting down any matches found.  This work is incredibly boring for the student worker, and is a prime candidate for automation (the same task is done over and over again, with a very specific set of criteria as output (match or no match).

Python’s Beautiful Soup library is built for exactly this kind of task – instead of having your student worker scan a bunch of web pages in your catalog, the script can do approximately the same thing by sending search terms to your catalog via URL, and returning back page elements that can tell you whether or not any matches were found.  In my example script, I’m using title and author elements, but you could modify this script to use other elements as long as they are indexed in your catalog – for example, you could send ISBNs, OCLC numbers, etc.

First, using Excel I concatenate a list of titles and authors with a domain and other URL elements to search my library’s catalog.  Here’s a few examples of what the URLs look like:

http://suncat.csun.edu/search~S9/X?SEARCH=t:(Los%20Angeles%20Two%20Hundred)+and+a:(Lavender)&searchscope=9&SORT=DX
http://suncat.csun.edu/search~S9/X?SEARCH=t:(The%20Land%20of%20Journeys'%20Ending)+and+a:(Austin)&searchscope=9&SORT=DX
http://suncat.csun.edu/search~S9/X?SEARCH=t:(Mathematics%20and%20Sex)+and+a:(Ernest)&searchscope=9&SORT=DX

I’ll save the full list of these (in my example, I have over 1000 titles and authors to check) in a plain text file called advancedtitleauth.txt.

Next, I start my Python script by calling the Beautiful Soup library, and some other libraries that are useful (urllib – a library built for fetching data by URLs; csv – a library for working with CSV files; and re, for working with regular expressions ).  You’ll probably have to install Beautiful Soup on your system first, which you can do if you have the pip Python package management system ³ installed by using sudo pip install beautifulsoup4 on your system’s command line.

from bs4 import BeautifulSoup
import urllib
import csv
import re

Then I create a blank array and define a CSV file into which to write the output of the script:

url_list = []

csv_out = csv.writer(open('output.txt', 'w'), delimiter = '\t')

The CSV file I’m creating will use tabs instead of commas as delimiters (hence delimiter = ‘\t’).  Typically when working with library data, I prefer tab-delimited text files over comma-separated files, because you never know when a random comma is going to show up in a title and create a delimiter where their should not be one.

Then I open my list of URLs, read it, append each URL to my array, and feed each URL into Beautiful Soup:

try:
  f = open('advancedtitleauth.txt', 'rb')
  for line in f:
    url_list.append(line)
    r = urllib.urlopen(line).read()
    soup = BeautifulSoup(r)

Beautiful Soup will go fetch the web page of each URL.  Now that I have the web pages, Beautiful Soup can parse out specific features of each page.  In my case, my catalog returns a page with a single record if a match is found, and a browsable index when a match is found (e.g., your title would be here, but it isn’t, so here’s some stuff with titles that would be nearby).  I can use Beautiful Soup to return page elements that tell me whether a match was found, and if a match is found, to write the permanent URL of the match for later evaluation.  This bit of code looks for an HTML div element with the class “bibRecordLink” on the page, which only appears when a single match is found.  If this div is present on the page, the script grabs the link and drops it into the output file.

try:
      link = soup.find_all("div", class_="bibRecordLink")
      directlink = str(link[0])
      directlink = "http://suncat.csun.edu" + directlink[36:]

In the code above, [36:] is Python’s way of noting the start position of a string – so in this case, I’m getting the end of the string starting with the 36th character (which in my case, is the bibliographic ID number of the item that allows me to construct a permalink).

If a title/author search results in multiple possible matches – that is, we might have multiple copies, or the title/author combo is too vague to land on just one item, the page that displays in our catalog shows a browsable list of brief record info.  In my script, I just grab the top result:

 try:
      briefcit = soup.find_all("span", class_="briefcitTitle")
      bestmatch = str(briefcit[0])
      sep = "&"
      bestmatch = bestmatch.split(sep, 1)[0]
      bestmatch = "http://suncat.csun.edu/" + bestmatch[39:]

In the code above, Beautiful Soup finds all the<span> elements with the class “briefcitTitle”, the script returns the first one, and again returns a URL stored in the bestmatch variable.

You can see a sample output of my lookup script here.  You can see that for each entry, I include publication information, direct links, or a best match link if the elements are found.  If none of the elements are found for a lookup URL, the line reads:

nopub nolink nomatch

We can now divide the output file into “no match” entries, direct links, or best match links.  Direct links and best match links will need to be double-checked by a student worker to make sure they actually represent the item we looked up, including the date and edition.  The “no match” entries represent titles we don’t have in our collection, so those can be evaluated more closely to determine if we want them.

The script certainly has room for improvement; I could write in a lot more functionality to better identify publication information, for example, to possibly reduce or eliminate the need for manual review of direct or partial matches.  But the return on investment for this script is fairly highfor a 37-line script written in an afternoon; we can re-use this dozens of times, and hopefully save countless hours of student worker boredom (and re-assign those student workers to more complex and meaningful tasks!).

Rudimentary Keyword Frequency Analysis

This second example involves, again, dealing with a task that could be done manually, but can be done much more quickly with a script.

My university needed to submit data for the AASHE Sustainability Tracking, Assessment, and Rating System (STARS) Report (https://stars.aashe.org/), which requires the analysis of data from campus course offerings as well as research output by faculty.  To submit the report, we needed to know how many courses we offer in sustainability (defined by AASHE “in an inclusive way, encompassing human and ecological health, social justice, secure livelihoods and a better world for all generations”) and how many faculty do research in sustainability.  This project was broken up into two components:  Analysis of research data by faculty and analysis of course data.

Sustainability Keywords

Before we even started analyzing research or course data, we needed to define an approach to identify what counts as “sustainability.”  Thankfully, there was precedent from the University of North Carolina, which had developed a list of sustainability-related keywords used to search against faculty research output ⁴  We adopted this list of keywords to lookup in faculty research articles and course descriptions.

Research data by faculty

We don’t have a comprehensive inventory of research done by faculty at our campus.  Because we were on a somewhat tight deadline to do the analysis, I came up with a very quick and dirty way of getting a lot of citations by using Web of Science.  Web of Science enables you to do a search for research published by affiliates of your university.  I was able to retrieve about 8,000 citations written by current or former faculty associated with my institution going back about 15 years.  Of course, we cannot consider the data in Web of Science to be fully representative of faculty research output, but it seemed like a good start at least.

Web of Science enables you to export 500-record chunks of metadata, so it took an hour or so to export the metadata in several pieces (see Figure 1 for my Web of Science export criteria).

Once I had all of the metadata for the 8,000 or so records written by faculty at my institution, I combined them into a single file.  Next, I needed to identify records that had sustainability keywords in either the title or abstract.

First, I created an array of all of the keywords, and turned that list into a Python set.  A Python set is different from a list in that the order of terms does not matter, and is ideal for checking membership of items in the set against strings (or in my case, a bunch of citation and abstract strings).

word_list = 'Agriculture,Alternative,Applied%Science [..snip..]'
word_set <span class="pl-k">=</span> <span class="pl-c1">set</span>(word_list.split(<span class="pl-s"><span class="pl-pds">'</span>,<span class="pl-pds">'</span></span>))

Note the % in “Applied%Science”.  For some reason my set lookup couldn’t match terms with spaces. My hacky solution was to replace spaces with % characters, and then do a find/replace in my spreadsheet of Web of Science data to replace all keyword matches with spaces (such as Applied Science) with percentage signs (Applied%Science).  Luckily, there were only 10 or so keywords on the list with spaces, so the find/replace did not take very long.  Note also that the set match lookup is case sensitive, so I actually found it easier to just turn everything to lower case in my Web of Science spreadsheet and match on the lower case term (though I kept both upper and lower case terms in my lookup set).

Then I checked to see if any words were in the title, abstract, or both, and constructed my query so that a new column would be added to an output spreadsheet indicating *which* matches were found:

for row in csv_reader:
 if (set(row[23].split()) & word_set) & (set(row[9].split()) & word_set) :
 csv_out.writerow(["title & abstract match",row[61],row[1],row[9],row[23],(set(row[9].split()) & word_set), (set(row[23].split()) & word_set)])

If any of the words in my set were found in the 23rd cell of the spreadsheet (the abstract) and the 9th cell of the spreadsheet (the title), then a row would be written to an output sheet indicating that sustainability keywords were found in the title and abstract, pulling in some citation details about the article (including author names), as well as a cell with a list of the matches found for both title and abstract fields.

I did similar conditionals for rows that found, for example, just a title match, or just an author match:

 elif set(row[9].split()) & word_set:
      csv_out.writerow(["title match",row[61],row[1],row[9],row[23], (set(row[9].split()) & word_set)])
    elif set(row[23].split()) & word_set:
      csv_out.writerow(["abstract match",row[61],row[1],row[9],row[23], (set(row[23].split()) & word_set)])

And that is pretty much the whole script!  With the output file, I did have to do a bit more work to identify current faculty at my institution, but I basically used the same set matching method above using a list provided by HR.

Because the STARS report also required analysis of courses related to sustainability, I also created a very similar script to lookup key terms found in course titles and descriptions.

Of course, just because a research article or course description has a keyword, or even multiple keywords, does not mean it’s relevant at all to sustainability.  One of the keywords identified as related to sustainability, for example, is “invest”, which basically meant that almost every finance class returned as a match.  Manual work was required to review the matches and weed out false positives, but because the keyword matching was already done and we could easily see what matches were found, this work was done fairly quickly.  We could, for example, identify courses and research articles that only had a single keyword match.  If that single keyword match was something like “sustainability” it was likely a sustainability-related course and would merit further review; if the single keyword match was something like “systems” it could probably be weeded out.

As with my author/title lookup script, if I had a bit more time to fuss with the script, I could have probably optimized it further (for example, by assigning weight to more sustainability-related keywords to help calculate a relevance score).  But again, a short amount of time invested in this script saved a huge amount of time, and enabled us to do something we would not otherwise have been able to do.

Python Resources

If you’re interested in learning more about Python and its syntax, and don’t have a lot of Python experience, a good (free) place to start is Google’s Python Class, created by Nick Parlante for Google (I actually took a similar class several years ago, also created by Dr. Parlante, through Coursera, which looks to still be available).  If you want to get started using Python right away and don’t want to have to fuss with installing it on your computer, you can check out the interactive course How to Think Like a Computer Scientist created by Brad Miller and David Ranum at Luther College.  For more examples of usage in Python for library work, check out Charles Ed Hill, Heidi Frank, and Mark Pernotto’s Python chapter in the just-released LITA Guide The Librarian’s Introduction to Programming Languages, edited by Beth Thomsett-Scott (full-disclosure: I am a contributor to this book).

Keeping up with technical skills and finding time to learn new things can be a struggle, no matter your role in a library (or in any organization, for that matter).  In some academic libraries, professional development opportunities have been historically available to librarians and library faculty, and less available (or totally unavailable) for staff positions.  In this post, I argue that this disparity, where it may exist, is not only prima facie unfair, but can reduce innovation and willingness to change in the library.  If your library does not have a policy that specifically addresses training and professional development for all library staff, this post will provide some ideas on how to start crafting one.

In this post, when referring to “training and professional development,” I mostly have in mind technology training – though a training policy could cover non-technical training, such as leadership, time management, or project management training (though of course, some of those skills are closely related to technology).

Rationale

In the absence of a staff training policy or formal support for staff training, staff are likely still doing the training, but may not feel supported by the library to do so.  In ACRL TechConnect’s 2015 survey on learning programming in libraries, respondents noted disparities at their libraries between support for technical training for faculty or librarian positions and staff positions.  Respondents also noted that even though support for training was available in principle (e.g., funding was potentially available for travel or training), workloads were too high to find the time to complete training and professional development, and some respondents indicated learning on their own time was the only feasible way to train.   A policy promoting staff training and professional development should therefore explicitly allocate time and resources for training, so that training can actually be completed during work hours.

There is not a significant amount of recent research reflecting the impact of staff training on library operations.  Research in other industries has found that staff training can improve morale, reduce employee turnover and increase organizational innovation.¹  In a review of characteristics of innovative companies, Choudhary (2014) found that “Not surprisingly, employees are the most important asset of an organization and the most important source of innovation.” ²  Training and workshops – particularly those that feature “lectures/talks from accomplished persons outside the organization” are especially effective in fostering happy and motivated employees ³ – and it’s happy and motivated employees that contribute most to a culture of innovation in an organization.

Key Policy Elements

Time

Your policy should outline how much time for training is available to each employee (for example, 2 hours a week or 8 hours a month).  Ensuring that staff have enough time for training while covering their existing duties is the most challenging part of implementing a training policy or plan.  For service desks in particular, scheduling adequate coverage while staff are doing professional development can be very difficult – especially as many libraries are understaffed.  To free up time, an option might be to train and promote a few student workers to do higher-level tasks to cover staff during training (you’ll need to budget to pay these students a higher wage for this work).  If your library wants to promote a culture of learning among staff, but there really is no time available to staff to do training, then the library probably needs more staff.

A training policy should be clear that training should be scheduled in advance with supervisor approval, and supervisors should be empowered to integrate professional development time into existing schedules.  Your policy may also specify that training hours can be allocated more heavily during low-traffic times in the library, such as summer, spring, and winter breaks, and that employees will likely train less during high-traffic or project-intensive times of the year.  In this way, a policy that specifies that an employee has X number of training hours per month or year might be more flexible than a policy that calls for X number of training hours per week.

Equipment and Space

Time is not enough.  Equipment, particularly mobile devices such as iPads or laptops – should also be available for staff use and checkout. These devices should be configured to enable staff to install required plugins and software for viewing webinars and training videos.   Library staff whose offices are open and vulnerable to constant interruption by patrons or student workers may find training is more effective if they have the option to check out a mobile device and head to another area – away from their desk – to focus.  Quiet spaces and webinar viewing rooms may also be required, and most libraries already have group or individual study areas.  Ensure that your policy states whether or how staff may reserve these spaces for training use.

Funding

There are tons of training materials, videos, and courses that are freely available online – but there are also lots of webinars and workshops that have a cost that are totally worth paying for.  A library that offers funding for professional development for some employees (such as librarians or those with faculty status), but not others, risks alienating staff and sending the message that staff learning is not valued by the organization.  Staff should know what the process is to apply for funding to travel, attend workshops, and view webinars.  Be sure to write up the procedures for requesting this funding either in the training policy itself or documented elsewhere but available to all employees.  Funding might be limited, but it’s vital to be transparent about travel funding request procedures.

An issue that is probably outside of the scope of a training policy, but is nonetheless very closely related, is staff pay.  If you’re asking staff to train more, know more, and do more, compensation needs to reflect this. Pay scales may not have caught up to the reality that many library staff positions now require technology skills that were not necessary in the past; some positions may need to be re-classed.  For this reason, creating a staff training policy may not be possible in a vacuum, but this process may need to be integrated with a library strategic planning and/or re-organization plan.  It’s incredibly important on this point that library leadership is on board with a potential training policy and its strategic and staffing implications.

Align Training with Organizational Goals

It likely goes without saying that training and professional development should align with organizational goals, but you should still say it in your policy – and specify where those organizational goals are documented. How those goals are set is determined by the strategic planning process at your library, but you may wish to outline in your policy that supervisors and department heads can set departmental goals and encourage staff to undertake training that aligns with these goals.  This can, in theory, get a little tricky: if we want to take a yoga class as part of our professional development, is that OK?  If your organization values mindfulness and/or wellness, it might be!

If your library wants to promote a culture of experimentation and risk-taking, consider explicitly defining and promoting those values in your policy.  This can help guide supervisors when working with staff to set training priorities.  One exciting potential outcome of implementing a training policy is to foster an environment where employees feel secure in trying out new skills, so make it clear that employees are empowered to do so.  Communication / Collaboration

Are there multiple people in your library interested in learning Ruby?  If there were, would you have any way of knowing?  Effective communication can be a massive challenge on its own (and is way beyond the scope of this post), but when setting up and documenting a training policy staff, you could include guidance for how staff should communicate their training activities with the rest of the library.  This could take the form of something totally low-tech (like a bulletin board or shared training calendar in the break room) or could take the form of an intranet blog where everyone is encouraged to write a post about their recent training and professional development experiences.  Consider planning to hold ‘share-fests’ a few times a year where staff can share new ideas and skills with others in the library to further recognize training accomplishments.

Training is in the Job Description

Training and professional development should be included in all job descriptions (a lot easier said than done, admittedly).  Employees need to know they are empowered to use work time to complete training and professional development.  There may be union, collective bargaining, and employee review implications to this – which I certainly am not qualified to speak on – but these issues should be addressed when planning to implement a training policy.  For new hires going forward, expect to have a period of ‘onboarding’ during which time the new staff member will devote a significant amount of time to training (this may already be happening informally, but I have certainly had experiences as a staff member being hired in and spending the first few weeks of my new job trying to figure out what my job is on my own!).

Closing the Loop:  Idea and Innovation Management

OK, so you’ve implemented a training policy, and now training and professional development is happening constantly in your library.  Awesome!  Not only is everyone learning new skills, but staff have great ideas for new services, or are learning about new software they want to implement.  How do you keep the momentum going?

One option might be to set up a process to track ideas and innovative projects in your library.  There’s a niche software industry around idea and innovation management that features some highly robust and specialized products (Brightidea, Spigit  and Ideascale are some examples), but you could start small and integrate idea tracking into an existing ticket system like SpiceWorks, OSTicket, or even LibAnswers.  A periodic open vote could be held to identify high-impact projects and prioritize new ideas and services.  It’s important to be transparent and accountable for this – adopting internally-generated ideas can in and of itself be a great morale-booster if handled properly, but if staff feel like their ideas are not valued, a culture of innovation will die before it gets off the ground.

Does your library have a truly awesome culture of learning and employee professional development?  I’d love to hear about it in the comments or @lpmagnuson.

Notes

[Editor’s Note:  This post is part of a series of posts related to ACRL TechConnect’s 2015 survey on Programming Languages, Frameworks, and Web Content Management Systems in Libraries.  The survey was distributed between January and March 2015 and received 265 responses.  The first post in this series is available here.]

In our last post in this series, we discussed how library programmers learn about and develop new skills in programming in libraries.  We also wanted to find out how library administrators or library culture in general does or does not support learning skills in programming.

From anecdotal accounts, we hypothesized that learning new programming skills might be impeded by factors including lack of access to necessary technologies or server environments, lack of support for training, travel or professional development opportunities, or overloaded job descriptions that make it difficult to find the time to learn and develop new skills.  While respondents to our survey did in some cases indicate these barriers, we actually found that most respondents felt supported by their administration or library to develop new programming skills.

Most respondents feel supported, but lack of time is a problem

The question we asked respondents was:

Please describe how your employing institution either does or does not support your efforts to learn or improve programming or development skills. “Support” can refer to funding, training, mentoring, work time allocation, or other means of support.

The question was open-ended, enabling respondents to provide details about their experiences.  We received 193 responses to this question and categorized responses by whether they overall indicated support or lack of support.  74% of respondents indicated at least some support for learning programming by their library administration, while 26% report a lack of support for learning programming.

Of those who mentioned that their administration or supervisors provide a supportive environment for learning about programming, the top kind of support mentioned was training, closely followed by funding for professional development opportunities.  Flexibility in work time was also frequently mentioned by respondents.  Mentoring and encouragement were mentioned less frequently.

However, even among those who feel supported in terms of funding and training opportunities, respondents indicated that time to actually complete training or professional development, is, in practice, scarce:

Work time allocation is a definite issue – I’m the only systems librarian and have responsibilities governing web site, intranet, discovery layer, link resover, ereserve system, meeting room booking system and library management system. No time for deep learning.

Low staffing often contributes to the lack of time to develop skills, even in supportive environments:

They definitely support developing new skills, but we have a very small technology staff so it’s difficult to find time to learn something new and implement it.

Respondents indicated the importance to their employers of aligning training and funding requests with current work projects and priorities:

I would be able to get support in terms of work time allocation, limited funding for training. I’m limited by external control of library technology platforms (centrally administrated), need to identify utility of learning language to justify training, use, &c.

26% of respondents indicate a lack of support for learning programming

Of those respondents who indicated that their workplace is not supportive of programming professional development or learning opportunities, lack of funding and training was the most commonly cited type of support that respondents found lacking.

Lack of  Funding and Training

The main lack of support comes in the form of funding and training. There are few opportunities to network and attend training events (other than virtually online) to learn how to do my job better. I basically have to read and research (either with a book or on the web) to learn about programming for libraries.

Respondents mentioned that though they could do training during their work hours, they are not necessarily funded to do so:

I am given time for self-education, but no formal training or provision for formal education classes.

Lack of Mentoring / Peer Support

Peer support was important to many respondents, both in supportive and unsupportive environments.  Many respondents who felt supported mentioned how important it was to have colleagues in their workplace to whom they can turn to get advice and help with troubleshooting.  Comments such as this one illustrate the difficulty of being the only systems or technology support person in one’s workplace:

They are very open to supporting me financially and giving me work time to learn (we have an institutional license to lynda.com and they have funded off site training), but there is not a lot of peer support for learning. I am a solo systems department and most of our campus IT staff are contractors, so there is not the opportunity for a community of colleagues to share ideas and to learn from each other.

Understaffing / Low Pay for Programming Skills

Closely related to the lack of peer support, respondents specifically mentioned that being the only technical staff person at their institution can make it difficult to find time for learning, and that understaffing contributes to the high workload:

There’s no money for training and we are understaffed so there’s no time for self-taught skills. I am the only non-Windows programmer so there’s no one I can confer with on programming challenges. I learn whatever I need to know on the fly and only to the degree it’s necessary to get the job done.

I’m the only “tech” on site, so I don’t have time to learn anything new.

One respondent mentioned that pay for those with programming skills is not competitive at his or her institution:

We have zero means for support, partially due to a complex web of financial reasons. No training, little encouragement, and a refusal to hire/pay at market rates programming staff.

Future Research and Other Questions

As with the first post in this series, the analysis of the data yields more questions than clear conclusions.  Some respondents indicated they have very supportive workplaces, where they feel like their administration and supervisors provide every opportunity to develop new skills and learn about the technologies they want to learn about.  Others express frustration with the lack of funding or ability to collaborate with colleagues on projects that require programming skills.

One question that requires a more thorough examination of the data is whether those whose jobs do not specifically require programming skills feel as supported in learning about programming as those who were hired to be programmers.  30% of survey respondents indicated that programming is *not* part of their official job duties, but that they do programming or similar activities to perform job functions.  Initial analysis indicates there is no significant difference between these respondents and respondents as a whole.  However, there may be differences in support based on the type of position one has in a library (e.g., staff, faculty, or administration), and we did not gather that information from respondents in this survey.  At least two respondents, however, indicates that this may be the case at least at some libraries:

Training & funding is available; can have release time to attend; all is easier for librarians to obtain than for staff to obtain which is sad since staff tend to do more of the programming

Some staff have a lot of support, some have nill, it depends on where/what project you are working on.

In the next (and final) post in this series, we’ll explore some preliminary data on popular programming languages in libraries, and examine how often library programmers get to use their preferred programming languages in their work.

[Editor’s Note:  This post is part of a series of posts related to ACRL TechConnect’s 2015 survey on Programming Languages, Frameworks, and Web Content Management Systems in Libraries.  The survey was distributed between January and March 2015 and received 265 responses.  A longer journal article with additional analysis is also forthcoming.  For a quick summary of the article below, check out this infographic.]

Our survey on programming languages in libraries has resulted in a mountain of fascinating data.  One of the goals of our survey was to better understand how staff in libraries learn about programming and develop their coding skills.  Based upon anecdotal evidence, we hypothesized that library staff members are often self-taught, learning through a combination of on-the-job learning and online tutorials.  Our findings indicate that respondents use a wide variety of sources to learn about programming, including MOOCs, online tutorials, Google searches, and colleagues.

Are programming skills gained by formal coursework, or in Library Science Master’s Programs?

We were interested in identifying sources of programming learning, whether that involved course work (either formal coursework as part of a degree or continuing education program, or through Massive Online Open Courseware (MOOCs)).  Nearly two-thirds of respondents indicated they had an MLS or were working on one:

When asked about coursework taken in programming, application, or software development, results were mixed, with the most popular choice being 1-2 classes:

However, of those respondents who have taken a course in programming (about 80% of all respondents) AND indicated that they either had an MLS or were attending an MLS program, only about a third had taken any of those courses as part of a Master’s in Library Science program:

Resources for learning about programming

The final question of the survey asked respondents, in an open-ended way, to describe resources they use to learn about programming.  It was a pretty complex question:

Please list or describe any learning resources, discussion boards or forums, or other methods you use to learn about or develop your skills in programming, application development, or scripting. Please includes links to online resources if available. Examples of resources include, but are not limited to: Lynda.com, MOOC courses, local community/college/university course on programming, Books, Code4Lib listserv, Stack Overflow, etc.).

Respondents gave, in many cases, incredibly detailed responses – and most respondents indicated a list of resources used.  After coding the responses into 10 categories, some trends emerged.  The most popular resources for learning about programming, by far, were courses (whether those courses were taken formally in a classroom environment, or online in a MOOC environment):

To better illustrate what each category entails, here are the top five resources in each category:

By far, the most commonly cited learning resource was Stack Overflow, followed by the Code4Lib Listserv, Books/ebooks (unspecified) and Lynda.com.  Results may skew a little toward these resources because they were mentioned as examples in the question, priming respondents to include them in their responses.  Since links to the survey were distributed, among other places, on the Code4Lib listserv, its prominence may also be influenced by response bias. One area that was a little surprising was the number of respondents that included social networks (including in-person networks like co-workers) as resources – indeed, respondents who mentioned colleagues as learning resources were particularly enthusiastic, as one respondent put it:

…co-workers are always very important learning resources, perhaps the most important!

Preliminary Analysis

While the data isn’t conclusive enough to draw any strong conclusions yet, a few thoughts come to mind:

• About 3/4 of respondents indicated that programming was either part of their job description, or that they use programming or scripting as part of their work, even if it’s not expressly part of their job.  And yet, only about a third of respondents with an MLS (or in the process of getting one) took a programming class as part of their MLS program.  Programming is increasingly an essential skill for library work, and this survey seems to support the view that there should be more programming courses in library school curriculum.
• Obviously programming work is not monolithic – there’s lots of variation among those who do programming work that isn’t reflected in our survey, and this survey may have unintentionally excluded those who are hobby coders.  Most questions focused on programming used when performing work-related tasks, so additional research would be needed to identify learning strategies of enthusiast programmers who don’t have the opportunity to program as part of their job.
• Respondents indicated that learning on the job is an important aspect of their work; they may not have time or institutional support for formal training or courses, and figure things out as they go along using forums like Stack Overflow and Code4Lib’s listserv.  As one respondent put it:

Codecademy got me started. Stack Overflow saves me hours of time and effort, on a regular basis, as it helps me with answers to specific, time-of-need questions, helping me do problem-based learning.

TL;DR?  Here’s an infographic:

In the next post, I’ll discuss some of the findings related to ways administration and supervisors support (or don’t support) programming work in libraries.