Rapid Prototyping Mobile Augmented Reality Applications

This Fall semester the Undergraduate Library at the University of Illinois at Urbana Champaign along with partners from the Graduate School of Library and Information Science and Computer Science graduate students with experience in programming OpenCV, will begin coding an open source mobile Augmented Reality (AR) app for deeper in-library engagement with both print and physical resources. The funding comes from a recently awarded IMLS Sparks! Grant. Our objectives include the following:

  • Create shelf recognition software for mobile devices that integrate print and digital resources into the on-site library experience and experiment with location based recommendation services.
  • Investigate the potential of creating a system that shows users how they are physically navigating an “idea space.”
  • Complete iterative rapid use studies of mobile software with library patrons and communicate results back to programming staff for incremental app design.
  • Work with our Library IT staff to identify skills and technical infrastructure needed in order to make AR an ongoing part of technology in libraries.
  • Make available the AR apps through the Library’s mobile labs experimental apps area (http://m.library.illinois.edu/labs.asp).

There are multiple problems with access to the variety of collections in our networked era (Lee, 2000) including their highly disparate nature (many vended platforms serving licensed library content) and their increasing intangibility (the move to massively electronic, or e-only access in libraries and information centers). Moreover, library collection developers are faced with the challenge of providing increased access to digital while still maintaining print. Lee (2000) argues for library research redefining library collections as information contexts.

This work will address the contextual information needs of library users while leveraging recent advances in mobile-networked technologies, experimenting with a way to increase access to collections of all types. The research team will deploy, test, and evaluate mobile applications that create novel “augmented book stacks.”

 

(a.) Subject of book stack set is identified by app index, and displayed on interface. (b.) Recommendations (e-book, digital items, or databases) brought onto interface in real-time. (c.) Popular books are indicated on title using circulation data from integrated library system historical circulation count (this can be a Z39.50 call or a pre-loaded circulation report database).

 

To create such applications, researchers will make use of video functionality that augment shelves of interest to a user in the library stacks inserting interactive graphics through the video feed of a phone onto the physical book stack environment in real-time. As a comparison to current state of the art mobile AR apps, like the ShelvAR app in development at Miami University, the proposed system does not require 2D tags as targets on books, but rather uses a combination of computer vision software code for feature detection and optical character recognition (OCR) software to parse the text of titles, call numbers, and subjects on the book stacks. A prototype project for OCR running in Android can be implemented following this tutorial. Our research group does not propose a replication of the state of the art, but will implement a system that pushes forward the state of the art in innovation for research and learning with AR in library stacks.

The project team will experiment with overlaying relevant resources from other parts of the library’s collection such as the library’s licensed set of databases, other Internet based resources, or books that are relevant but not shelved nearby. This augmentation will enhance the serendipitous discovery of books so that items relevant to a user’s location, but not shelved near her can be brought into the browsing experience; with this technology books that are checked out, or otherwise unavailable can still be made useful to a users information search. Our staff will experiment with system features that create “idea spaces” for the user, which will serve to help students and library users exploit previous discovery routes and spaces in the book stacks. The premise of “idea spaces” comes from an unspoken assumption among librarians: the intellectual organization of items in library collections are valuable constructs. By presenting graphical overlays of the subject areas of the collection, we make this assumption explicit and assert that as a user navigates the geographic spaces of a library collection, they are actually navigating intellectual spaces. With a user location is paired an idea (or set of related ideas), delivered in our proposed system with a graphical overly in the video feed. The user’s location, her context in the collection, is the query point for the idea spaces system.

This experiment will be valuable for all libraries that support print and digital resources. Underscoring this work is the overarching concern with making all library collections more accessible. Researchers will undertake rapid prototyping (as a test case for the chosen method see: Jones & Richey, 2000) of the augmented reality feature set in order to understand user preferences of mobile interfaces that best support location-based recommendations, and make all results of this experimentation including software code and computing workflows freely available. Such experimentation could lead to profound changes in the way people research and learn in library spaces.

Grant activities will begin in October 2012 and conclude September 2013. The evaluation plan for the grant is a systematic measurement of project outputs against the stated goals with the resulting evaluative outputs communicating what worked and was useful for library patrons in AR apps. By operationalizing a rapid evaluation of augmented reality services the research team hopes to identify the fail points for mobile services in this domain in addition to the most desired and useful feature set for all augmented reality systems in library book stacks.

Cited

Jones, T. & Richey, R.  (2000) “Rapid Prototyping methodology in action: a developmental study,” Educational Technology Research and Development 48, 63-80.

Lee, H. (2000), “What is a collection?” Journal of the American Society for Information Science, 51 (12) 1106-1113.

Suggested Reading

Regarding collocation objectives in library science see: Svenonius, E. (2000), The Intellectual Foundation of Information Organization, MIT Press, Cambridge, MA, pp.21-22

See also

Additional sample code for image processing with Android devices available here, courtesy of openly available lecture notes from Stanford’s Digital Image Processing Course EE368.

Forthcoming this October, a paper detailing additional AR use cases in library services: Hahn, J. (2012). Mobile augmented reality applications for library services. New Library World 113 (9/10)


8 Comments on “Rapid Prototyping Mobile Augmented Reality Applications”

  1. Andromeda says:

    One: this is SUPER COOL.

    Two: will the code be available, e.g. on github, with an open license?

    • Jim Hahn says:

      Yes, I’m interested in making this openly available, GitHub is definitely in mind for continuing work after the grant ends. Our team thought that this module could eventually become a kind of plugin for any library to add to their mobile app, or if they don’t have a mobile app, it could also be a stand alone service that they customize locally from the developed architecture. It will be openly developed and end of project code will be in the U of I institutional repository for permanent access.

  2. [...] Posted: September 3, 2012 | Author: Jim Hahn | Filed under: academic librarianship, coding, data, library, mobile,technology | Tags: android, augmented reality, computer vision, Open CV, optical character recognition, rapid prototyping |3 Comments » [...]

  3. [...] Rapid Prototyping Mobile Augmented Reality Share this:TwitterFacebookLike this:LikeBe the first to like this. [...]

  4. This web site Is Truly wonderful and amazing information maybe there is and looking so captivating.. I look and feel good to generally be here.. Keep sharing good things with buddies.. Thanks meant for sharing…

  5. […] Mobile Augmented Reality Applications  […]