Members of the PRIMO Committee review and evaluate instructional materials submitted for inclusion into the PRIMO database according to the following criteria:
Criterion #1
The resource’s instructional design is pedagogically effective, i.e., it teaches well according to the scope and learning objectives stated by the submitter.
- The purpose and objectives are clearly stated.
- The resource’s organization supports the objectives.
- The content supports the objectives.
- Offers opportunities to utilize higher order thinking skills.
- Addresses various learning modalities (text, audio, video).
Criterion #2
The resource incorporates assessment tools and technique(s).
- Assessment of learning is linked to the stated learning objectives and utilizes formative and/or summative methods
- Learning is assessed in multiple places throughout the resource via methods which require students to utilize higher order thinking skills.
- The content of the assessment tool is aligned to the content of the resource.
- The assessment tools provide answers and instant constructive feedback for correct and incorrect answers. ideally, feedback for incorrect answers either links to or references pertinent areas to review.
- Users can retake the assessment to retest what they learned.
Criterion #3
The technology used to create the resource enhances the learning experience, i.e., is appropriate and effective.
- The chosen technology aligns with the resource’s learning outcomes/instructional strategies; it is used as a means to an instructional end and does not distract from the learning experience.
- The chosen technology works as intended and easy to use.
- The technology is compatible with different browsers and devices (including mobile options).
- Required plug-ins or downloads are easily obtained and easy to install.
- The design elements are well-executed and professional (e.g., sound and video have been edited to flow smoothly, interactive elements are easy to use, etc.).
Criterion #4
The content and language of the resource are clear, effective, and easy to use.
- Instructions and explanations are easy to follow.
- Language and content are appropriate to the learning objectives.
- Language and content are appropriate to the target audience.
- The design elements are appropriate to the target audience.
- Users can easily find their way back to the home page and/or to other sections, and navigation mechanisms are consistent from page to page.
- The text is easy to read.
Criterion #5
All information included within the resource is accurate.
- The resource does not contain typographical errors.
- There are no apparent factual errors.
- The resource appears to be relatively up-to-date and current.
- The resource offers some contact information (email or phone) for author and/or library if questions or technical problems arise.
Criterion #6
The resource adheres to W3C’s Web Content Accessibility Guidelines.
- There is an index, table of contents, or site map to facilitate navigation, and each page has a descriptive title.
- Images are accompanied by detailed text alternatives.
- Captions or transcripts are provided for audio and video content.
- Enough time is provided to read and use content. Audio and video elements must have player controls that allow the user to pause or navigate forward and backward.
- There is a visible and logical sequence or structure to content, including tab sequence on keyboard operation.
- Instructions and content do not rely on image characteristics such as color, shape, or size for imparting information.
- Interactive elements can be controlled by the keyboard in addition to or instead of the mouse.
Criterion #7
The resource demonstrates unique or creative use of graphics, examples, interactive elements such as programmed feedback and flexible learning paths, and other supporting elements.
- Contains interactive elements that require the user to interact with the tutorial beyond watching, clicking through, and/or reading.
- Interactivity is interspersed throughout the resource at relevant points of instruction.
- Effectively incorporates design elements such as graphics, multimedia, and/or flexible learning paths.
- The design elements show evidence of creativity and originality; they are not tired copies of material from other learning objects.
- The design elements contribute to the coherence of the resource.
- The developer’s approach to teaching or to the use of technology is thought-provoking; it stimulates ideas about ways to communicate with learners.
Criterion #8
The resource is relevant, available and accessible to those outside of the developer’s institution and presents a model for other developers.
- The resource is available to people outside of the developer’s institution. Ideally, the resource is open source or has a Creative Commons or equivalent license.
- The resource, or any part of the resource, does not need authentication or special permissions in order be accessed by users outside of the institution.
- The structure (e.g., chunking, sequencing, transitions, connections, reinforcement, assessment, feedback) can be adapted to teaching other skills, resources, or ideas.
- The method of presentation (e.g., use of text, sound, graphics, animation, video, language, layout, pacing, examples) can be adapted to teaching other skills, resources, or ideas.
How submissions are evaluated:
Reviewers assign each of the criteria statements an agreement value using the following scale:
- 5 = Meets all the components covered in the description.
- 4 = Meets all but one of the components covered in the description. The missing component does not affect the working of the resource.
- 3 = Meets many (missing more than one) of the components covered in the description. The missing components do not significantly affect the project overall on this criterion.
- 2 = Meets many (missing more than one) of the components covered in the description. The missing components do significantly affect the project overall on this criterion.
- 1 = Meets only one component for the criteria and is lacking enough components that the resource is affected.
- 0 = Does not satisfy any components described. After scoring each of the seven criteria, the values are totaled.
After scoring each of the eight criteria, the values are totaled. Review scores from all reviewers are averaged. Submissions receiving an average score of 32 or higher are added to the PRIMO database.
A note on the project scope…
For PRIMO, quality is a greater goal than is comprehensiveness. However, in order to maintain a quality database of easily searchable materials, PRIMO will not accept the following for review:
- entire suites of multiple modules/tutorials (authors are welcome to submit modules separately for review, if the technology used permits)
- projects that use technologies or methodologies already well represented in the database
- simple pathfinders that do not instruct
- promotional materials that advertise services or facilities but do not instruct
- company or vendor-produced sites
- projects targeted at a K-12 audience
- projects that primarily consist of material from other institutions
- basic videos or screen captures
- projects in languages other than English
For further explanation of what the committee looks for with regard to the criteria please see PRIMO What to Know Before You Submit.
A note on legal matters…
The IS PRIMO Committee does not make any claims or guarantees whatsoever as to the accuracy or currency of the data represented on this site.
Peer-Reviewed Instructional Materials Online (PRIMO) database
