Tim Gulden & Jonas Siegel Center for International and Security Studies at Maryland (CISSM)

1. The development of prototype data management systems for the Biological Research Security System (BRSS) Tim Gulden & Jonas Siegel Center for International and Security Studies at Maryland (CISSM)

2. Oversight process goals • Prevent the inadvertent and the deliberate creation of biological agents that are more transmissible and/or virulent than those presently known. • Match the system's disclosure requirements with the degree of risk involved in research, thereby setting information disclosure standards.  • Facilitate the licensing of life science researchers and institutions. • Facilitate the peer-review of biological research to assess the security implications of specific experiments

3. Data management system design goals • Prioritize transparency, reliability, integrity, and security (e.g. use of open-source software and HTTPS protocol). • Maintain flexibility, scalability, and low cost. • Ensure ease of use for researchers, reviewers, and administrators. • Permit local users to adapt system for multiple uses.

4. Types of questionnaires • New • Automatically generated to collect basic information when a new object is created • Examples:  Projects, Users, Institutions, Labs • Triggered • Generated by answers to questions in other questionnaires • Allowed for the number of questions to be minimized • Examples: Security measures, Recombinant materials, Human subjects, Animal use, Bio-safety • Parameterized • Generated for each choice from a list of choices • Same set of questions can be asked for each pathogenic organism to be used.

5. Specific questionnaires • Licensing questionnaires • New user - The first step in the licensing process. Required for anyone who wants access to the system. • New institution - Required to be completed only once for each institution; can be updated over time. • New laboratory - Required to be completed only once for each laboratory; can be updated over time. • Security measures - An in-depth look at security measures that builds off questions in "new institution" or "new lab" forms.

6. Specific questionnairescont... • Project questionnaires • New project -  Each experiment submitted for review requires a separate "new project" form. • Pathogenic organisms - Required if an experiment includes pathogenic organisms. • Recombinant materials - Required if an experiment includes recombinant materials. • Human subjects; Animal use; Laboratory biosafety& engineering controls - Available if an institution wants to simultaneously collect information for a separate review board.

7. Features of two prototype designs • BRSS prototype • Developed by independent contractors • Built using Python • Designed to support mandatory participation • Accommodates on-line and in-person review • Multi-node capability • Semi-automated process • EoC portal  • Adapted from BRSS prototype in collaboration with Berkeley • Built using Plone • Voluntary participation • Accommodates on-line review only • Intuitive graphic user interface  • No nested questionnaires • Process not automated

8. Testing the prototype systems • In-person peer review of BRSS prototype (January 2005) • CISSM EoC portal trials (November 2009-February 2010)

9. Lessons learned

10. Experience with University of Maryland researchers • Relatively few researchers and officials who engaged with our oversight system were interested in having it manage all of the information collection necessary for the spheres of bio-security, bio-safety, human subjects, animal use, lab administration, etc. • Researchers acknowledged that the BRSS software and review process could complement existing arrangements by assessing the implications of sensitive experiments.

11. Researchers will accept regulation, but won't seek voluntary review • A voluntary system provides few incentives for researchers to submit experiments for review and for reviewers to dedicate time to the review process. • Institutions also have few incentives to permit affiliated researchers to participate.  • Even if a researcher wants to participate, he or she may be constrained by institutional policies and/or national laws.

12. Refined functionality and design are needed to ensure efficient participation and review • A system that matches disclosure requirements with the degree of risk involved in a research proposal requires an  automated set of questionnaires that adapts to researcher and reviewer inputs.  • Readily available, secure communication channels are needed between system participants, including the researchers who submit experiments for review. • User interface matters.

13. Decision Rules • Multiple decision rules are possible: Consensus; Majority; Majority with Veto; or Appeal. • Most reviews do not involve up or down decisions • Reviewers can suggest modifications. • Reviewers can question potential dangers associated with a particular experiment. • Much of the value of the review process comes from researchers thinking through and answering the questions.

14. Decision Process • Whether the decision process is entirely online or has a face-to-face component should impact system design. • Implications with face-to-facecomponent • A limit to number of experiments that can be reviewed. • An inability to request additional information from researchers in a timely fashion. • Decision needs to be recorded in system separately.

15. Decision processcont. • Implications if only online • Reviewers should each record their opinion after thorough online discussion. • System can implement decision rule (e.g. consensus). • Resolving differences of opinion and reaching consensus among reviewers may be particularly challenging because of constraints on communication. • Online review lacks the immediacy of in-person review, which could lead to a review process dragging on. • Excellent user interface is critical.