1 / 14

Dewey Peer Review System

Dewey Peer Review System. Cal Collins and Shai Sachs Isovera Consulting Dec. 2006. Overview. The Peer Review System is an add-on to the Dewey Cataloging System which supports the peer review process for a collection of learning objects.

siusan
Télécharger la présentation

Dewey Peer Review System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dewey Peer Review System Cal Collins and Shai Sachs Isovera Consulting Dec. 2006

  2. Overview • The Peer Review System is an add-on to the Dewey Cataloging System which supports the peer review process for a collection of learning objects. • All learning objects contributed to the BEN Portal must be peer-reviewed. Currently, many collections conduct peer review “off-line” or via email. • The Dewey Peer Review System will allow your editors and reviewers to work collaboratively online, with no need to maintain many different versions of Excel spreadsheets, Word documents, etc. Internet consulting for non-profits

  3. Features • Major areas of functionality are: • Creation of user roles to facilitate targeted workflow assignments • Workflow management • Review form management • Group reviews, single-person reviews, assignment, feedback • System monitoring • Features which are not included: • Segmented workflows and resource grouping • Templated emails • Unassignments and rollbacks Internet consulting for non-profits

  4. Technical Components • Minimum requirements: • MySQL 4.0 • PHP 4.0 • Apache 1.3 • Works on Windows and Unix servers • Designed as a plug-in to the Dewey system. • Plug-ins are mini-applications supported by the Cake PHP framework. • Cake PHP is the framework which supports the Dewey Cataloging System. Internet consulting for non-profits

  5. Timeline • First draft of software requirement specification has been released. • Final draft of software requirement specification due for release in early Jan 2007. • Review form management will be released in March 2007. • Performing reviews, assignments, and feedbacks released in May. • Possible pilot demonstration of the system in May 2007. • Workflow management and system monitoring released in June. • Beta release commences in June 2007. • Production release in July 2007. Internet consulting for non-profits

  6. Workflow Management • All resources in the system are reviewed through the workflow. • The workflow is a chronological series of steps designed by the administrator. It may be revised over time. • The first step is always the submission of the resource. • Subsequent steps are either: • Assignments: one user orders one or more other users to carry out some other step • Parallel reviews: a set of reviews conducted by one or more people • Single-person reviews: a review conducted by exactly one person Internet consulting for non-profits

  7. Single-person reviews • A single-person review results in a decision on the resource. • Available decisions include: • Accept • Reject • Feedback / Accept with revisions • Redirect the resource to another step in the process Internet consulting for non-profits

  8. A sample workflow Assignment by Managing Editor Submitted Assignment by Associate Editor Parallel Reviews by Peer Reviewers Single-Person Review by Associate Editor Accepted Feedback from Author Rejected Internet consulting for non-profits

  9. A sample workflow, Contd User Groups: Managing Editors, Associate Editors, Peer Reviewers • Submitted • Assignment step: “Assignment by Managing Editor” • From: Managing Editors • To: Associate Editors • Assignment step: “Assign peer reviews” • From: Associate Editor • To: • Lab exercise reviewers, if the resource is a Lab exercise • Textbook reviewers, if the resource is a Textbook • Parallel Review step: “Perform peer reviews” • Lab exercise review • Review form: Lab exercise Review Form • Textbook review • Review form: Textbook Review Form • Single-Person review step: “Issue final decision” • Available redirection steps: “Peer Review” Internet consulting for non-profits

  10. Review Forms • Review forms are the key to substantive quality assessments. They ask reviewers questions like “Is the resource scientifically accurate?”, “Please comment on the originality of the resource.”, and “What is the pedagogical merit of the resource?” • Review forms are designed by the administrator and may be revised over time. • Review forms consist of a single page of inputs. Internet consulting for non-profits

  11. Review Form Inputs • Available input types are single-line text, multiple-line text, drop-down lists, and files. • When designing a drop-down list input, administrators specify a controlled vocabulary which contains the options for the drop-down list. • Inputs are accompanied by standard Dewey user interface cues: • Prompt (appears to the left of the input in bold) • Brief help (appears below the prompt, not in bold) • Extended help (opens in a separate window) Internet consulting for non-profits

  12. Review FormsThe Gory Truth • Review form management relies on the metadata management, controlled vocabulary management, and input process management capabilities provided by Dewey. • Review form management provides simplified, streamlined access to these capabilities. • Each review form corresponds to a single metadata structure, a single input process, and as many controlled vocabularies as are needed to support the various drop-down lists within the form. • So administrators and reviewers get all the power of the Dewey cataloging system and none of the limitations of the BEN-LOM. Internet consulting for non-profits

  13. Feedback • A Word document version of the specification is available; you can make comments and suggest revisions directly on the document using Track Revisions. (Tools  Track Revisions) • You may also send suggestions and feedback directly to ssachs@isovera.com. • While we will try to accommodate as many requested revisions as possible. The workflow management system has been carefully designed to avoid inconsistencies within the system, “matrix management” of the review process, and other potential pitfalls of peer review systems. Some requests may have to be modified to preserve consistency. Internet consulting for non-profits

  14. Conclusion and Next Steps • Read through the first draft of the specification. • Analyze whether the workflow management, as specified, will facilitate your review process. • Analyze whether the review form management system will allow you to design the kinds of review forms that you want your reviewers to complete. • If not, tell us why not. If you have questions, email ssachs@isovera.com and we will try to provide clarifications. Internet consulting for non-profits

More Related