PRIMO Site of the Month: March 2018
Intro Tutorials for Engineers
http://researchguides.uvm.edu/engineering-tutorials
Author: Graham Sherriff
Institution: The University of Vermont Libraries
Interviewee: Graham Sherriff
Interviewer: Marcia Rapchak
Description (provided by the author):
“Intro Tutorials for Engineers” is a suite of interactive tutorials created in 2017 to support engineering seniors at the University of Vermont. These tutorials provide seniors taking capstone courses in Civil, Environmental, Electrical, and Mechanical Engineering with foundational instruction in key types of text-based information. Each tutorial defines a type of publication and its characteristics, explains its value for engineering research, and presents key resources and search strategies. This instruction becomes the basis for in-depth research assignments. The tutorials were created in the Guide on the Side platform, which integrates authentic, real-time web content and offers opportunities for interactive learning.
Q: These tutorials contain specialized content for engineering students. How did you determine the subject matter for the tutorials? How did you develop the learning outcomes?
A: I created the first iterations of these tutorials to support senior-level engineering design courses at the University of Vermont (UVM). Our capstone courses in Electrical, Mechanical, and Civil & Environmental Engineering require student teams to do an extensive research assignment at an early stage of their project’s design phase.
A significant number of students were starting these courses with little prior experience working with some key publication types for engineering and technology. Thus, the learning outcomes and content needed to go from beginning to end, covering the sequence of “why-where-how”: the importance and value of a type of publication (“why”), where to search for it (“where”), and strategies for finding relevant material (“how”).
Q: How long did it take for you to create all the tutorials from proposal to launch? Who was involved in the creation and what roles did they play?
A: The tutorials went through a process of analysis, definition of needs and outcomes, design, development, testing, peer review, revision, and evaluation. (Appropriate, I think, for tutorials that support design courses!) For each tutorial, the entire process took 30-40 hours.
UVM engineering faculty Dustin Rand and John Lens, who teach the capstone courses, had an essential role in green-lighting the project, testing the tutorials, and reviewing their effectiveness. Also, their TAs were instrumental in implementing the tutorials within the courses and grading the students’ work.
I’m also grateful for insightful feedback from librarian colleagues at UVM and members of the Engineering Libraries Division (ELD) of American Society for Engineering Education (ASEE), who provided peer review of the tutorials.
Q: What made you decide to use the Guide on the Side software? Did you have any difficulties using it?
A: I think Guide on the Side is a highly effective means of online instruction. It enables students to learn through engagement with authentic, real-time web content. It’s an active and autonomous mode of learning. It’s a fairly robust platform for scoring quizzes, and student feedback is usually positive.
I was already familiar with Guide on the Side’s features and capabilities, as I’m the administrator/developer for other Guide on the Side tutorials that UVM uses to flip some foundational information literacy instruction for our first-year students. Another virtue of Guide on the Side is it’s a simple program and easy to use.
Q: Some of the tutorials use two windows while others use one. What determined this approach? Have you had any difficulties with students using the two windows?
A: Guide on the Side uses frames to display text and questions on the left side and web content on the right. By default, it displays the two frames in a single window. But, for security reasons, some websites, such as Google Patents and National Technical Reports Library, are configured to prevent their content from being embedded in another site’s frame. In these cases, it was necessary to use the two-window option.
I think the single-window default is preferable for the user. It’s easier to manage a single window and it avoids obstacles like blocks on pop-ups. But using both the single-window and two-window models didn’t prompt any reports of confusion from our students; they seemed to use both set-ups without significant problems.
Q: How did you determine how much written instruction and explanation you should use in each tutorial? How did you balance providing students with enough information while not overwhelming them?
A: It’s certainly hard to find that balance. The tutorials were created under the assumption that a user might come to them with negligible prior knowledge, so there was a lot of information to cover. But I also wanted the students to learn as much as possible through their own discovery. My goal was to craft questions that gave the students opportunities to learn through exploration and experimentation.
I aimed to minimize cognitive load through good “writing for the web” practices: minimizing text, chunking content, keeping material above the fold, and highlighting key points. And I tried to sustain engagement by breaking up expository text with a learner question or task every 2-3 tutorial pages.
Q: You have several questions throughout each tutorial to check learning. How did you determine what questions to ask? What were your major goals in asking the questions? How did you determine whether a question was too hard?
A: The questions in the main part of the tutorial are formative questions, designed to aid or reinforce learning. My goal was to provide opportunities for students to test or apply the concepts explained in the tutorial. A secondary goal was to sustain their interest and engagement.
The quiz at the end is a summative assessment. I designed the questions to measure comprehension of key concepts and accomplishment of the learning outcomes. Test results showed that these questions were harder than the formative questions, but not unreasonably hard.
Q: How have you advertised or deployed the tutorial? Who is using it and how?
A: Our students in Electrical, Mechanical, and Civil & Environmental Engineering capstone courses completed the tutorials in fall 2017. In spring 2018, I was able to adapt some of them for a first-year Civil & Environmental course with a literature review assignment.
Q: Can you talk about your partnership with the Engineering department? How did you manage broad buy-in so quickly?
A: I had been working with the Electrical, Mechanical, and Civil & Environmental Engineering capstone courses for several years, so the partnerships with faculty were already in place. But we had been using the “one-shot” model for research instruction. The course instructors and I agreed that this model was suboptimal; a 45-minute presentation was clearly not enough time to cover all the necessary information about all the relevant types of publication and not an effective mode of learning. There was no available time in the schedules for class sessions, so the faculty and I saw flipping my instruction with online tutorials as a logical option. (We retained the in-class presentation, but refocused it on the reinforcement of key points and the clarification of questions and uncertainties.) I was able to apply the tutorials in the first-year Civil Engineering course so quickly because the course instructor was the same professor who had adopted them in the Civil Engineering capstone course. He was familiar with their content and saw how some of the material (with edits) would benefit the first-year course, where students do a smaller research assignment.
Q: Have you collected assessment results from the quizzes? If so, what have the results shown you? Have you used the assessment to make any changes to the tutorials?
A: Professor Rand, Professor Lens, and I have reviewed the students’ research reports and observed notable year-on-year improvements in the quality of their research process, the authority and relevancy of their sources, and the subsequent revision of their projects.
Also, aggregated data from the tutorial quizzes have provided some insights. For example, scores on the “Intro to Patents” tutorial were significantly lower than the others, suggesting that students found this topic more difficult. And the question scores for that topic highlight specific concepts related to patent searching that evidently were challenging for many students. I will use this feedback to revise the tutorials for the next academic year. They need to be reviewed periodically in any case because the external web content is liable to change.
There’s scope for creating additional tutorials on other types of engineering publications (dissertations and theses, handbooks, datasets). However, my priority for the time being is enhancing and extending the tutorials that most directly support the capstone assignments. For example, I’m considering a tutorial on the distinctions between standards, codes, regulations, and laws, which can be a point of confusion for students.
Q: In future tutorials, will you make any changes to your approach? What have you learned from this process that you will carry over into your next tutorial?
A: I think the sequence of “why-where-how” and the organizational principle of alternating exposition and questioning have been effective. At first, I worried the students might think the tutorials were too formulaic, but testing and peer review reassured me that the consistent structure, and the familiarity it offers, was a positive.
In future tutorials, I’d like to continue working on question design. It’s quite straightforward to create a puzzle-solving-type question, where the student is given precise directions and required to find the one correct answer. I think there’s potential for greater learning from questions that are more open-ended and exploratory, but these are harder to design.
