This Is How I (Attempt To) Work

Editor’s Note: ACRL TechConnect blog will run a series of posts by our regular and guest authors about The Setup of our work. The first post is by TechConnect alum Becky Yoose.

Ever wondered how several of your beloved TechConnect authors and alumni manage to Get Stuff Done? In conjunction with The Setup, this is the first post in a series of TechConnect authors, past and present, to show off what tools, tips, and tricks they use for work.

I have been tagged by @nnschiller in his “This is how I work” post. Normally, I just hide when these type of chain letter type events come along, but this time I’ll indulge everyone and dust off my blogging skills. I’m Becky Yoose, Discovery and Integrated Systems Librarian, and this is how I work.

Location: Grinnell, Iowa, United States

Current Gig: Assistant Professor, Discovery and Integrated Systems Librarian; Grinnell College

Current Mobile Device: Samsung Galaxy Note 3, outfitted with an OtterBox Defender cover. I still mourn the discontinuation of the Droid sliding keyboard models, but the oversized screen and stylus make up for the lack of tactile typing.

Current Computer:

Work: HP EliteBook 8460p (due to be replaced in 2015); boots Windows 7

Home: Betty, my first build; dual boots Windows 7 and Ubuntu 14.04 LTS

eeepc 901, currently b0rked due to misjudgement on my part about appropriate xubuntu distros.

Current Tablet: iPad 2, supplied by work.

One word that best describes how you work:

Panic!
Don’t panic. Nothing to see here. Move along.

What apps/software/tools can’t you live without?

Essential work computer software and tools, in no particular order:

  • Outlook – email and meetings make up the majority of my daily interactions with people at work and since campus is a Microsoft shop…
  • Notepad++ – my Swiss army knife for text-based duties: scripts, notes, and everything in between.
  • PuTTY – Great SSH/Telnet client for Windows.
  • Marcedit – I work with library metadata, so Marcedit is essential on any of my work machines.
  • MacroExpress and AutoIt – Two different Windows automation apps: MacroExpress handles more simple automation (opening programs, templating/constant data, simple workflows involving multiple programs) while AutoIt gives you more flexibility and control in the automation process, including programming local functions and more complex decision-making processes.
  • Rainmeter and Rainlander – These two provide customized desktop skins that give you direct or quicker access to specific system information, functions, or in Rainlander’s case, application data.
  • Pidgin – MPOW uses both LibraryH3lp and AIM for instant messaging services, and I use IRC to keep in touch with #libtechwomen and #code4lib channels. Being able to do all three in one app saves time and effort.
  • Jing – while the Snipping Tool in Windows 7 is great for taking screenshots for emails, Jing has proven to be useful for both basic screenshots and screencasts for troubleshooting systems issues with staff and library users. The ability to save screencasts on screencast.com is also valuable when working with vendors in troubleshooting problems.
  • CCleaner – Not only does it empty your recycling bin and temporary files/caches, the various features available in one spot (program lists, registry fixes, startup program lists, etc.) make CCleaner an efficient way to do housekeeping on my machines.
  • Janetter (modified code for custom display of Twitter lists) – Twitter is my main information source for the library and technology fields. One feature I use extensively is the List feature, and Janetter’s plugin-friendly set up allows me to highly customize not only the display but what is displayed in the list feeds.
  • Firefox, including these plugins (not an exhaustive list):

For server apps, the main app (beyond putty or vSphere) that I need is Nagios to monitor the library virtual Linux server farm. I also am partial to nano, vim, and apt.

As one of the very few tech people on staff, I need a reliable system to track and communicate technical issues with both library users and staff. Currently the Libraries is piggybacking on ITS’ ticketing system KBOX. Despite being fit into a somewhat inflexible existing structure, it has worked well for us, and since we don’t have to maintain the system, all the better!

Web services: The Old Reader, Gmail, Google Drive, Skype, Twitter. I still mourn the loss of Google Reader.

For physical items, my tea mug. And my hat.

What’s your workspace like?

Take a concrete box, place it in the dead center of the library, cut out a door in one side, place the door opening three feet from the elevator door, cool it to a consistent 63-65 degrees F., and you have my office. Spending 10+ hours a day during the week in this office means a bit of modding is in order:

  • Computer workstation set up: two HP LA2205wg 22 inch monitors (set to appropriate ergonomic distances on desk), laptop docking station, ergonomic keyboard/mouse stand, ergonomic chair. Key word is “ergonomic”. I can’t stress this enough with folks; I’ve seen friends develop RSIs on the job years ago and they still struggle with them today. Don’t go down that path if you can help it; it’s not pretty.
  • Light source: four lamps of varying size, all with GE Daylight 6500K 15 watt light bulbs. I can’t do the overhead lights due to headaches and migraines, so these lamps and bulbs help make an otherwise dark concrete box a little brighter.
  • Three cephalopods, a starfish, a duck, a moomin, and cats of various materials and sizes
  • Well stocked snack/emergency meal/tea corner to fuel said 10+ hour work days
  • Blankets, cardigans, shawls, and heating pads to deal with the cold

When I work at home during weekends, I end up in the kitchen with the laptop on the island, giving me the option to sit on the high chair or stand. Either way, I have a window to look at when I need a few seconds to think. (If my boss is reading this – I want my office window back.)

What’s your best time-saving trick?

Do it right the first time. If you can’t do it right the first time, then make the path to make it right as efficient  and painless as you possibly can. Alternatively, build a time machine to prevent those disastrous metadata and systems decisions made in the past that you’re dealing with now.

What’s your favorite to-do list manager?

Post it notes on a wall
The Big Picture from 2012

I have tried to do online to-do list managers, such as Trello; however, I have found that physical managers work best for me. In my office I have a to-do management system that comprises of three types of lists:

  • The Big Picture List (2012 list pictured above)- four big post it sheets on my wall, labeled by season, divided by months in each sheet. Smaller post it notes are used to indicate which projects are going on in which months. This is a great way to get a quick visual as to what needs to be completed, what can be delayed, etc.
  • The Medium Picture List – a mounted whiteboard on the wall in front of my desk. Here specific projects are listed with one to three action items that need to be completed within a certain time, usually within one to two months.
  • The Small Picture List – written on discarded Choice review cards, the perfect size to quickly jot down things that need to be done either today or in the next few days.

Besides your phone and computer, what gadget can’t you live without?

My wrist watch, set five minutes fast. I feel conscientious if I go out of the house without it.

What everyday thing are you better at than everyone else?

I’d like to think that I’m pretty good with adhering to Inbox Zero.

What are you currently reading?

The practice of system and network administration, 2nd edition. Part curiosity, part wanting to improve my sysadmin responsibilities, part wanting to be able to communicate better with my IT colleagues.

What do you listen to while you work?

It depends on what I am working on. I have various stations on Pandora One and a selection of iTunes playlists to choose from depending on the task on hand. The choices range from medieval chant (for long form writing) to thrash metal (XML troubleshooting).

Realistically, though, the sounds I hear most are email notifications, the operation of the elevator that is three feet from my door, and the occasional TMI conversation between students who think the hallway where my office and the elevator are located is deserted.

Are you more of an introvert or an extrovert?

An introvert blessed/cursed with her parents’ social skills.

What’s your sleep routine like?

I turn into a pumpkin at around 8:30 pm, sometimes earlier. I wake up around 4:30 am most days, though I do cheat and not get out of bed until around 5:15 am, checking email, news feeds, and looking at my calendar to prepare for the coming day.

Fill in the blank: I’d love to see _________ answer these same questions.

You. Also, my cats.

What’s the best advice you’ve ever received?

Not advice per se, but life experience. There are many things one learns when living on a farm, including responsibility, work ethic, and realistic optimism. You learn to integrate work and life since, on the farm, work is life. You work long hours, but you also have to rest whenever you can catch a moment.  If nothing else, living on a farm teaches you that no matter how long you put off doing something, it has to be done. The earlier, the better, especially when it comes with shoveling manure.

Security!

“I didn’t do anything! All I did was plugged in the USB stick to see if there was a name in any documents so I can return it to its owner.”

“I kept getting pop-ups on my workstation, and I keep clicking the cancel button on all of them. Why won’t they stop popping up?”

“Six people have called in the last half hour, saying they couldn’t access The Expensive Electronic Resource. I’ve called the vendor, and they said there’s been strange activity on our IP address and their system’s not allowing us to access their site…”

Have you heard any of the above, or have others to add? If so, you’re not alone. When you’re one of the techies in an academic library, you are on the front line when things go wrong. You help people get through printing, emails, and various library systems troubleshooting, and you’re good at it. How good are you, though, in regards of dealing with library IT security?

Why bother with security? What’s at stake and what to do about it
Parody error message for an OPAC
Source: Credaro,A.B.(2002). Computer Error Messages for Library OPACs. Warrior Librarian Weekly [online]
http://www.warriorlibrarian.com/LOL/portal.html

I mean, who is that desperate to break library IT security? All we have is bibliographic records, and, really, who in the heck wants them?

The reality is that academic libraries have much to offer for those who want to break in and wreak havoc, including student and faculty data, restricted resources, and access to the campus network. And yes, there are bots out there that screen scrape MARC records from OPACs that slow systems down to a crawl when a bot is scraping away (I’ve been through my share, and they’re a pain to deal with). Though academic libraries usually have the benefit of campus IT to take care of antivirus and firewall setup and maintenance, it is up to the library staff themselves to ensure that their system is secure.

The most important thing you can do is to be proactive. Assessing loopholes in your library technology setup before an attack will not only decrease the ways that your system can be compromised but also decrease the damage done to your system if your system is compromised.

Passwords
Your campus IT has password requirements built into various campus systems including hardware; library systems usually do not see the same treatment. You also have systems not forcing password changes after a certain time. And, since these systems typically not to talk to each other, you have staff using the same password for multiple accounts. Do any of the systems or applications you use in your library have any built-in password requirements? Can the system be set to automatically require a password change after a certain amount of time? If the systems in question cannot do either of the above, you can still create password policies that will need to be manually enforced.

System logs and usage reports
System and report monitoring can help pinpoint suspicious activity as well as determine if a system has been compromised. Many of you may remember Aaron Swartz systematically downloading materials from JSTOR on the MIT network in 2011 1. Sometimes unauthorized access to a library resource happens with one person systematically downloading a huge number of materials; other times, like University of Saskatchewan found out when they looked at their reports, unauthorized access may be dispersed geographically 2. In similar situations regular monitoring of usage reports would tip off library staff of the unusual behavior and contact the vendor to relay the information before the vendor’s systems cuts off access to all users.

Servers also need monitoring for suspicious activity. If your library is responsible for its own servers, there are many server monitoring applications to choose from, like Nagios 3. In addition to monitoring server resources through these applications, depending on the server setup you will have access to a variety of system logs for your perusal. I occasionally see a bot unsuccessfully to hack into one of our servers while scanning through our system logs; however, that’s the only way I would have known about those attempts. Logs and reports might be your first sign that your system has been compromised, so it’s best to check them regularly.

Human staff

A van with "free candy" painted on its side
Any email from “Campus IT” asking for your password = the above picture. Source: https://secure.flickr.com/photos/stephen_nomura/3070133859/

The biggest security loophole in any IT environment is your average human. Humans plug in USB sticks left behind in the computer lab into their workstation, they download files from emails or websites, and they keep clicking on those flashing pop-ups. Humans are also too trustworthy – you’ve probably seen the email where “Campus IT” is asking for your password so they can increase your storage quota. Many people still email their passwords and other sensitive information because they truly believe that the email is from IT, their bank, that businessman overseas, and so on. The best way to close the human loophole is through training. Training library staff in security issues can take on many forms. For example, at our monthly library staff meeting at Grinnell College we dedicate 10-15 minutes for “Tech Topics” where we regularly cover security topics, including what to do when you think your computer is infected, passwords, and data security. Staff have access to resources covered at these meetings in our shared drive for future reference.

Unfortunately, you cannot completely close the human loophole. While you can control the staff side, you cannot prevent a student giving out their password to their friends, or a faculty member giving their password to a colleague at a different institution. Not all is lost – tightening other loopholes does help with dealing with the user loophole.

Where to start

There’s a lot to keep track of when you are tackling IT security at your library; you might feel overwhelmed, not knowing where to start. Here are a few places and resources to help you start:

Campus IT: Most likely your campus IT department already has campus-wide policies on various topics, including password changes, what standards 3rd party vendors must meet when storing institutional data on off campus servers, and what to do when you suspect a networked computer is infected with a virus. Read the policies and talk to your campus IT staff to see how you can adapt their policies in your library’s specific needs.

SEC4LIB: Blake Carver, of LISNews fame, has created an online resource dedicated to library IT security. The website has a number of resources as well as a wiki with some outlines covering general IT security issues. If you find yourself with a library IT security question, there is a listserv where like-minded library staff can point you in the right direction.

Here!: Does your library have a security policy or action plan? Do you have a security horror story that you want others to learn from your mistake? Share them in the comments below.

  1. Schwartz, John. “Open-Access Advocate is Arrested for Huge Download.” New York Times, Jul 20, 2011. http://search.proquest.com/docview/878013667?accountid=7379.
  2. White, Heather Tones. “Electronic Resources Security: A look at Unauthorized Users.” Code4Lib Journal 12 (December 2010): http://journal.code4lib.org/articles/4117.
  3. Silver, T. Michael. “Monitoring Network and Service Availability with Open-Source Software.” Information Technology & Libraries 29, no. 1 (March 2010): 8-22.

A Brief Trip into Technology Planning, Brought to You By Meebo

The Day That Meebo Died

Today is the day that many librarians running reference services dreaded – Meebo discontinuing most of their products (with the exception of the Meebo Bar). Even though Meebo (or parts of it) will still live on in various Google products, that still doesn’t help those libraries who have build services and applications around a product that has been around for a while (Meebo was established in 2005).

If Meebo was any indication, even established, long running technology services can go away without much advanced notice. What is a library to do with incorporating third party applications, then? There is no way to ensure that all the services and applications that you use at your library will still be in existence for any length of time. Change is about the only constant in technology and it is up to us who deal with technology to plan for that change.

How to avoid backing your library into a corner with no escape route in sight

The worst has happened – the application you’re using is no longer being supported. Or, in a more positive light,  there’s a new alternative out there that performs better than the application your library is currently using at the moment.  The scenarios above have different priorities; migration due to discontinuation of support will probably happen on a faster timeline than upgrading to a better application. Overall, you should be prepared to survive without your current 3rd party applications with minimal amount of content loss and service disruption. For this post I’ll be focusing on third party application support and availability. Disruptions due to natural disasters, like fire, flooding, or, in Grinnell’s case, tornadoes, is equally important, but will not be covered at length in this post.

Competition (or lack there of)

When news broke that Google purchased Meebo, most weren’t sure about what would be next for the chat service. Soon afterwards, Meebo gave a month’s notice about the discontinuation of most of their products. Fortunately, alternative chat services were plentiful. Our library, for example, subscribes to LibraryH3lp, but we were using Meebo Messenger as well as the MeeboMe widget for some course pages to supplement LibraryH3lp’s services. After the announcement, our library quickly switched the messenger with Pidgin, and are working on replacing the Meebo widgets with LibraryH3lp’s widgets.

Having a diverse, healthy pool of different applications to choose from for a particular service is a good place to be when the application you use is no longer supported. Migrations are never fun, but consider the alternative. If you’re using a service or application that does not have readily available alternatives, how will your services be affected when that application is no longer supported?

The last question wasn’t rhetorical. If your answer is looking at a major service disruption, especially to services that are deemed by your library as mission-critical, then you’re putting yourself and the library in a precarious position. The same goes if the alternatives out there require a different technical skill set from your library staff. Applications that require a more advanced technical skill set will require more training and run the heightened risk of staff rejection if the required skill level is set too high.

Data wants to be backed up

Where’s your data right now? Can you export it out of the application? Do you even know if you can export your data or not? If not, then you’re setting yourself up for a preventable emergency. Exporting functionality and backups are especially important for services that are living outside of your direct control, like a hosted service. While most hosted services have backup servers to prevent loss of customer data, you should still have the ability to export your data and store it outside of the application. It’s best practice and gives you the peace of mind that you do not have to recreate years’ worth of work to restore data lost due to vendor error or lack of export functionality.

Another product that is widely used by academic libraries, LibGuides, provides a backup feature where you can export your guides in XML or individual guides in HTML. It will take some work for formatting and posting the data if needed, but the important thing is that you have your data and you can either host it locally in case of emergencies or harvest the content when the time comes to move on to another application.

Some technology service audit questions

Here are some general questions to start you down the path of evaluating where your library currently stands with third party applications you rely on for providing specific library services. Don’t worry if you find yourself not as prepared as you want to be. It’s better to start now than when you learn that another application you use will be shutting down.

  • What third party applications does your library currently use to provide library services?
  • Are there other comparable services/applications available?
    • What training resources are available for alternative applications?
    • What technical skills do these applications require? Are they compatible with the technical skills found with the majority of library staff?
  • Which applications are used for mission-critical library services?
  • Can you export your data and/or settings from the application?
    • If so, how often is the data being exported?
    • Where is the backup file stored? Locally? Remotely?
  • What is the plan if the application…
    • …is no longer supported?
    • …goes offline due to a service disruption?
      • …for a couple of hours?
      • …longer than a day?
      • …during finals week/first week of the semester/midterms (high pressure/high stakes times for library users)?

While there are many potential landmines when using third party applications for library services, these applications overall help expand and provide user services in various ways. Instead of becoming a technological recluse and shunning outside applications, use these applications wisely and make sure that your library has a plan in place.

How to make peace with error messages

At last, after all those hours toiling under the glow of the computer screen, your first script is completed. All those hours learning to code have finally paid off. Holding your breath, you enter the command to execute the script… only to have an error message appear on the screen. You shake your fist to the sky and curse whatever deities you believe in, but the error message remains unchanged, almost like it is staring into your soul…

Error messages != fun, or the Art of Debugging

Error messages happen to everyone. The causes of error messages vary; sometimes the error is caused by a bug hidden in the system, other times the error stems from the human typing at the keyboard. In every case, error messages can be frustrating and time consuming (For those of you who have hunted for hours to find that missing closing bracket in your script, you know the pain that I speak of). Error messages are here to stay no matter how thorough you are with your code, so you’ll need to know how to deal with them in an efficient manner.

Making bugs visible

The first step of debugging your script is to make sure that you’re actually receiving error messages when things go wrong. Depending on the language you use, you can change your error settings to output errors in a log, in the command window, on a web page, etc. You can also change how detailed the error messages are or even create custom error messages to help pinpoint where in the script the code is failing. If your script is not running and you don’t see any errors pop up, your best bet is to look at the documentation for that programming language for error reporting.

Speaking of documentation…

RTFM

For the purposes of this post, RTFM means “Read the Friendly Manual”. Most programming languages have various documentations available in both online and print formats. The documentation is a good place to start when you suspect that the error is caused due to syntax errors or a function that is not being used properly.

Example: You have a python script that is returning the following error when you run the script [1]

>>> while True print 'Hello world'
  File "<stdin>", line 1, in ?
    while True print 'Hello world'
                   ^
SyntaxError: invalid syntax

The  “SyntaxError: invalid syntax” tells you that your code is not formatted correctly and you have a little carrot pointing to the last letter on the print function, so one would assume that the error involves the use of print in the while statement in some way. A quick search on the python website shows the proper syntax for constructing the while statement:

while True: print 'Hello world'

Note that we were missing a colon between the expression (while True) and the suite (print ‘Hello world’)

Reading the documentation is also a good way to reduce errors, so it’s worth your time to seek out good resources about the language you’re using and study them before and while you’re building the script.

GIYF

GIYF – Google is your friend. Librarians have a love-hate relationship with Google for reasons that have been covered extensively elsewhere. Google, however, is a staple in the programmer’s life. Copy the error message, paste it in the search box, and you’ll receive a multitude of hits of varying quality. If I do an exact phrase search on the (very common) PHP error “Parse error: syntax error, unexpected T_VARIABLE”, Google comes back to me with around 82,200 results. That’s a lot of results to wade through. A good thing to remember is that the same criteria that many librarians teach their user in evaluating sites for research can be applied while seeking out help with troubleshooting errors. By applying what is taught in many Information Literacy sessions, you will quickly narrow down the number of sites to use in your bug squashing process.

Once you have done a few searches for errors in one language, you hopefully will have found a few web sites that are consistent in providing detailed, accurate information about troubleshooting errors.  Some sites and resources that have been useful when I tracked down various error messages:

Some library-related resources for those dealing with errors while working on various library modules/script libraries in various programming languages:

There are many other places where you can search for help which you will find in your error tracking time.

Now for some audience participation:

  • Do you have a resource that helped you with errors and bugs?
  • What errors have you come across with coding for library related projects, and how did you find a fix?

Please share in the comments below. Happy bug squashing!

 

[1] Error example from http://docs.python.org/tutorial/errors.html#syntax-errors

 

Workflow Automation in Technical Services: Part 2

Note: This is part two of a two part series on workflow automation in Technical Services. Part one covered the what and process of workflow automation and an example of an item level workflow automation process. Part two will discuss batch level workflow automation and resources/tools for workflow automation.

Last time, we discussed the basics of workflow automation and some examples of item-level automation in cataloging and acquisitions workflows. Automating workflows on an item-to-item basis provides greater consistency and efficiency in daily tasks done by staff, allowing them to spend more time on more complex workflows and tasks that may not be so readily automated. Item level workflow automation can be a low barrier investment in creating a more efficient operation.

Then you have the electronic journals, ebooks, and databases. You have large record files that are tied to physical resources – for example, record downloads from WorldCat Cataloging Partners. And then there are all those records in the system – MARC, XML, whatnot – that have missing or incorrect information (the infamous “dirty data”). Why can’t we just stick with item-level processing for everything?

Item level automation or batch automation?

For item level automation, you have a very granular level of control over the process, dealing with items one at a time. If the items are very similar in nature or have only a couple differences in how each item will be processed, though, then going through each item individually probably doesn’t make a lot of sense. On the other hand, batch processing allows you to go through many items at once, which makes adding or maintaining resources a quicker job than going through item by item. You do give up a certain level of control over details with batch processing, however, which leaves you to decide where the “good enough” marker should go in terms of data quality.

Overall, you want to avoid sub-optimizing your workflow. Sub-optimization happens when a part of an organization focuses the success of its own area instead of the entire organization’s success [1]. Going through each resource record individually might give you the greatest control over the record, but if you’re going through a file containing 10,000+ records individually, even with an item level automated workflow, the turnaround time for creating access for all those resources will be much higher than if the file was processed at once. However, with the right tools, you can deal with record batches with speed and a good level of control over the data.

MarcEdit is your friend

Many people have at least heard about MarcEdit, or have colleagues who have used it extensively. MarcEdit is a freely available program (for Windows) created by Terry Reese that works with MARC records in a variety of ways. You can add, delete, or modify fields in records, create MARC records from data in spreadsheets, crosswalk to and from the MARC format, split files, join files, generate call numbers, de-duplicate records – and that’s only part of what you can do with MarcEdit. Also, if you find yourself going through the same batch workflow for the same files on a regular basis, MarcEdit’s Script Wizard helps with automating routine batch processing workflows.

Example: Missing 041 1_ subfield h, or, this item is a translation, not in two languages!

Many of you may have moved your older library catalogs to a newer discovery layer; I’ve survived one move at my previous place of work and will probably have another move under my belt soon. One consequence of moving to a new discovery layer is that data previously ignored by the previous layer sticks out like a sore thumb in the new layer. This example is one of those dirty data discoveries: a particular MARC variable field incorrectly indicated that an item is in two or more languages instead of a translation. Not only you have unhappy library users who thought you had a copy of The Little Prince in both French and English, but this error exists in a few thousand records, finding yourself with a potentially resource intensive cleanup project.

If you can isolate and export those records in one (or a couple of) files from your database, then you can use MarcEdit to clean up the field in a relatively short time. Open the file in MarcEdit’s MarcEditor, and make your way to the “Edit Subfield” under the tools menu. Let’s say that there are a lot of records that have engfre in the 041 field and you want to change all the records with that entry at once. Replace the engfre field data with eng$hfre and you’ve taken care of all those records in one pass.

Since you probably have more than engfre in your file, you can use regular expressions in MarcEdit to change multiple fields at once regardless of language code. Using the Find/Replace tool, search for the 041 field subfield a, but this time add your regular expression and mark the “Use regular expression” box. The following expression is assuming that the 041 field has two language codes that are three letters in length, so you will have to do a little cleanup after running this replace command to catch the three or more language codes as well as two letter language codes. (h/t to zemkat for the regular expression!)

Libraries and modules and packages, oh my!

What if you’ve been learning some code, or are looking for an excuse to learn? You’re in luck! Some of the common programming languages have tools to deal with MARC data. Rolling your own batch automation scripts and applications allows you the most flexibility in working with other library data formats as well. However, if you haven’t programmed before, choose smaller projects to start. In addition, if the script or application doesn’t work, you’re your own tech support.

Example: Creating order records for patron driven acquisition (PDA) items triggered for purchase

Patron driven acquisition usually involves the ingestion of several hundred to thousands of records into the local database for items that are not technically owned by the library at that point in time. Depending on the PDA vendor one uses, the item is triggered for purchase after it reaches a use threshold (for example, 10 page views). The library will receive an invoice with these purchases, but we will still need to create order records in the system to show that these items have been bought. Considering that on a given week,  the number of purchases can range from single digits to higher double digits, that’s a lot of order records to manually key in.

After dabbling with pymarc at code4lib 2010, I thought this would be a good project to learn more about pymarc and python overall. Here is an outline of the script actions:

  1. In the trigger report spreadsheet, extract the local control numbers for the items triggered for purchase.
  2. Execute a SQL query against the local database for our locally developed next generation catalog, matching the local control number and extracting the MARC records from database.
  3. In each MARC record:
  • add a 590 and 790 field for donor/fund information
  • add a 949 field containing bibliographic record overlay and the order record creation information for the system, including cost of the item extracted from the spreadsheet.
  • change the 947 field data to indicate that the item has been purchased (for statistical reporting later on)
  1. Write the MARC records to a file for import into the ILS.

The output file is then uploaded into the ILS manually, which gives staff the chance to address any issues with the records that the system might have before import. Overall, the process from downloading the trigger report spreadsheet to uploading the record file into the ILS takes a few minutes, depending on the size of the file.

Which automation tools and resources to use?

There are a multitude of other automation tools and resources that cannot be fully covered in two blog posts. Your mileage may vary with these tools; you might find Macro Express to be a better fit for your organization than AutoIt, or you find that working with ruby-marc is easier for you than MarcEdit (resource links listed below). The best way to figure out what’s right for you is to play around with various tools and get a feel for them. More often than not, you’ll end up using multiple tools for different levels and types of workflow automation.

Don’t forget about the built-in tools in existing applications as well! Sometimes the best tools for the job are already there for you to take advantage of them.

For your convenience, here are the tools mentioned in the two blog posts, including a few others:


[1] http://dictionary.cambridge.org/dictionary/business-english/sub-optimization

Workflow Automation in Technical Services: Part 1

Note: This is part one of a two part series on workflow automation in Technical Services. Part one will cover the what and process of workflow automation and an example of an item level workflow automation process. Part two will discuss batch level workflow automation and resources/tools for workflow automation.

The mysterious door at the library

Door leading into Technical Services
Photo by author

A majority of you might have passed by this door many times in your library lives. Sometimes it isn’t even a door; maybe a room divider, or an invisible line that runs across the room. In any case, you may have ventured into the space called “Technical Services” (or a similar name), but do you know what goes on there? For most libraries, Technical Services staff acquire, create, and maintain access to library materials, spanning from books and a box of rocks to various electronic databases and digitized local collections. Without them, it would be hard for a library to serve its users: no physical items to borrow, no electronic journals to search for articles, and no metadata in the library discovery layer for users and staff to search for those resources. With the variety of items come a variety of workflows to process those items, many of which are repeated at various intervals: some once a week while others repeated multiple times a day. Staff time and resources are spoken for every time a workflow is repeated. Every time a workflow is manually repeated, less time and resources can be spent on other projects or on new projects that would add value to existing collections or add new collections for library users to use. Technology provides a variety of strategies for workflow automation that reduce time spent on repetitive workflows.

What is workflow automation?

The oversimplified answer to this question is that workflow automation is the process where you have the computer do the things that it can be programmed to do, thereby reducing repetitive manual actions by the staff member.

There are two types of automation to consider when you look at your workflows:

  1. Data Entry: This type of automation is fairly straight forward, and you’ve probably already done this type of automation already without realizing it. For example, the automation script completes a form with data that remains the same for each form or types out standard text in an email being sent to a vendor. Useful for automating repetitive keystrokes, be it system codes, text, or even creating new documents in certain applications, such as an item recor. The automation script is hard-coded, meaning that the output of that script will be the same every time you run it.
  2. Decision Making: This type of automation makes all the decisions for you! Okay, while it won’t make every decision for you, several automation languages and programs can handle fairly complex decision making flowcharts using standard conditionals. For example, if bibliographic record “A” has field “B”, then do action ”C”; else do action “D”. As you probably already guessed, this type of automation resembles coding to a certain extent. The automation script that is designed to deal with several possible outcomes is not hard-coded like the data entry script described above.

What can be automated?

Most Technical Services departments acquire, create, and maintain access to a variety of different formats, from physical to electronic formats. Traditionally, workflows focus on the individual item going through the department and its various teams: acquisitions, cataloging, and processing, for example. With the changeover to electronic formats, workflows are going more towards a batch approach, processing and/or cataloging multiple items (for example, a collection of ebooks) at once.

In addition to adding materials to library collections, a library’s Technical Services staff do a fair amount of database maintenance for the library’s ILS (Integrated Library System). The term “dirty data” is thrown around the TS departments, covering database projects dealing with misspellings, outdated codes, or incorrect codes – anything that could inhibit a library user’s access to the resource.

Why should I automate my workflows?

  • Better quality control of workflow and data. Any time you let a human near a workflow, errors can be introduced into a workflow: incorrect codes, mistyped text, or mishandled items. Having an automated workflow cuts down on the workflow’s fail points and allow for better overall consistency and accuracy.
  • Save staff time.  You and your staff spend a good amount of time with repetitive keystrokes and decisions. Even small repetitive actions add up during the work day, resulting in hours of valuable staff time and resources. By automating the repetitive actions, you free up staff time to work on more complex workflows which are not as easily automated.

How do you decide what workflows to automate?

  • Flowchart your workflow.  A simple flowchart from the beginning of the workflow to the end might reveal several places where current manual decision making can be relegated to a script. If a person is currently looking for a code in the order record to figure out what location code they should enter in the item record, the script could be set to do the same.
  • What are the patterns? In each step, what data remains constant throughout all items? What codes, phrases, or fields do you insert every time you go through the workflow? Is there a pattern of going from one application to another at the same point in every workflow? One record to another?
  • How will the script access the data? Working with a file of MARC records will be different than working with a bibliographic record that is open in your ILS. Having a file of data is easier, but if you’re automating an item-level workflow, you will be dealing with windows that you have to work with. Getting data from a window can be tricky; sometimes you are able to access the data directly, and other times you will have to scrape the screen to get to the data that you want to work on with the script.

Example: Receipt Cataloging

At my former place of work, Technical Services had three levels of cataloging: receipt cataloging, copy cataloging, and original cataloging. All monographs would go through the receipt cataloging process, with items being bumped to the two higher levels of cataloging. The majority of items that go through receipt cataloging, having met a list of 40+ criteria, are fast-tracked to physical processing, shortening the time between the item arriving at the library to being placed on the shelf, which is the overreaching goal of receipt cataloging. The criteria range from determining if the record is DLC (Library of Congress) to determining if the 008, 050, and 260 ‡c dates match in the bibliographic record (if not a conference publication).

Given that the criteria and the decision making flowchart are fairly standard and straightforward, this workflow was built with automation in mind. My predecessor used Macro Express (ME) for the first version of the receipt cataloging macros. When we got to the point where we were bumping up against ME’s limits, I migrated the macros to AutoIt, where I was able to include many more quality control checks on the bibliographic and item records.

Below is a screencast where I walk through the receipt cataloging process. If I wasn’t explaining what was happening, the whole process would have taken a minute and 10 seconds to complete, a couple of seconds more if the item was bumped to another team in the department. Compared to a five minute turnaround time if our staff manually checked every criteria, the macros allows the department to go through more items during the day with better quality control.

Bonus Example: Ordering from GOBI

Another workflow at my former place of work involved ordering monographs from GOBI. The workflow, unlike receipt cataloging, have a lot more complex decision making flowchart and more exceptions. While I could not automate on the level of receipt cataloging, there were still patterns and routines that I could automate, such as searching the library catalog with information supplied by GOBI, and determining which codes to enter in the 949 field in the OCLC record (for exporting into our database).

Below is a screencast that shows a part of the notification ordering automation script set.

Preview for Part 2

In this post, I covered more of the item level workflow automation possibilities. More of Technical Services workflows, however, are changing towards dealing with many items at once. In part 2, I will discuss some examples of batch process automation and several tools (including those mentioned in this post) that can assist in making life easier in Technical Services.

I Want to Learn to Code, but…

"Coder Rage" by librarykitty, some rights reserved.

You may have seen people posting that they are learning to code with CodeYear, mentioned in our earlier blog post “Tips for Everyone Doing the #codeyear”.  While CodeYear and Codecademy are not the first sites to teach programming, CodeYear has seen quite a bit of marketing and notice, especially in the library world (#libcodeyear and #catcode).

Many find themselves, however, in a familiar situation when dealing with learning to code. And it starts with the person saying or thinking “I want to learn to code, but…

Do you fall under any of these categories?

1. “I don’t have enough time to learn coding.”

You can work through the time issue in two ways. The first way is block off time. You have to look at your schedule and decide, for example, “ok, I’m working on my coding lesson between 1 to 2pm.” Once you made that decision, tell the rest of the world, so that they know that you’re working on learning something during that time.

For some folks, though, blocking off an hour may be impossible due to disruptions from work or personal life. When you’re in a situation where frequent disruptions are a fact of life, documentation is your friend. Keep notes of what you learned, what questions you have, what issues you ran across, and so on – this will make sure that you do not end up having to repeat a lesson, or losing track of your thoughts during a lesson.

2.  “This is too hard.”

Here I must stress one of the key survival traits for people learning to code: ask questions! Find people who are taking the same lesson and ask. Find coders and ask. Find an online forum and ask. Post your question on Twitter, Facebook, blog, or any other broadcasting medium. Just ASK.

More often than not your question will be answered, or you will be pointed in the right direction in answering your question. The overused saying “there is no such thing as a stupid question” applies here. Coding is a community activity, and it’s to your benefit to approach it as such.

3. “I don’t like the tutorial/course.”

It’s OK to say “hey, this course isn’t what I thought it would be” or “hey, I’m not finding this course useful.” Ask yourself, “in which environment do I feel like I learn the most?” Is it a physical classroom? A virtual classroom? Do you like learning on your own? With a small group of friends? With a large group?  There are various formats and venues where you can find courses in coding, from credit-earning classes to how-to books. For example, the Catcode wiki lists a variety of coding lessons or learning opportunities at various levels of coding knowledge. Choose the one (or a few) that will fit best with you, and go for it. It might take a few tries, but you will find something that works for you.

So, if you find yourself saying “I want to learn code, but…,” there is hope for you yet.

Find what’s holding you back, tackle it, and work out a possible solution. If you don’t get it the first time, that’s OK. It’s OK to fail, as long as you learn and understand why it failed, and apply what you learned in future endeavors. For now, we are stuck in learning coding the hard way: practice, practice, practice.

Learning code the hard way, on the other hand, is not too hard once you have taken the first few steps.