Assessing Research Management at Canada’s National Research Council

1. Assessing Research Management at Canada’s National Research Council AEA/CES Conference 2005 Crossing Borders, Crossing Boundaries R&D Topical Interest Group – Presentation Toronto, October 28, 2005

2. Presentation Outline • About the National Research Council of Canada (NRC) • Challenges in Research Management • What is the Research Management Self-Assessment (RMSA) Tool? • Development of the Tool • Progress to Date, and Moving Forward

3. The National Research Council of Canada – Science at Work for Canada Economic value • Incubators and start-up support • Knowledge and technology transfer/diffusion • Commercialization and new company creation • SME support Government of Canada’s leading resource for scientific research, development and technology-based innovation 20 institutes across Canada, 2 technology centres, IRAP, CISTI • Innovation Systems and Support • Regional innovation systems • Information and knowledge networks • National codes and standards • Research and Development • Basic/fundamental to applied • Technology development • National R&D infrastructure • R&D training • International collaborations • Research areas include: • Biotechnology • Information and Communications Technologies • Aerospace • Manufacturing • Construction • Ocean engineering • Genomics • Fuel cells • Bioinformatics • High-performance computing • Photonics • Nanotechnology • Environmental and sustainable development technologies Resources (2004-05): Full time staff (FTE): 4,100+ Expenditures: $712.4M Appropriations spent: $653.0M Revenues: $75.2 million Infrastructure: 175 buildings, 517,407 m2

4. Our Research Environment

5. More generic Uncertainty about where most valuable discoveries lie Risks and costs associated with uncertainty Unpredictable results given the pursuit of the unknown Difficulty in assessing contribution/impact of research results Often long timeframes before outcomes or impacts of research become evident Others facing NRC Balancing creativity and accountability (value for Canada) Managing High Quality Personnel within federal framework Planning in a dynamic environment Prioritizing with multiple stakeholder interests Managing multiple collaborations Establishing attribution of results in collaborative efforts Balancing cluster activities, horizontal program requirements and institute research activities Some Challenges in Managing Research

6. What is RMSA? The Research Management Self-Assessment (RMSA) is a practical, diagnostic tool developed by NRC for use by its institutes in support of continuous improvement and effective management of research and research-related activities. The RMSA: • A framework and codified research management practices against which institutes can assess themselves • Complementary to NRC’s annual performance reporting, and program evaluation • Helps to identify links to other corporate resources and tools Furthermore, RMSA allows: • A common language across NRC • Knowledge sharing and exchange • A non-threatening, honest, open self-examination for institutes • Engagement of research and related staff • Opportunity for management development

7. RMSA Framework (August 2005)

8. RMSA Development

9. How Does RMSA Work? 1. Assessment Planning 4. Follow-up Action Monitor progress against action plan, and re-assess as needed for other categories over time Planning and implementation of follow-up action – follow-up actions to be determined based on prioritized opportunities from assessment results Establishing the institute-specific context – anticipated issues / challenges; participants; timing; communications… Key areas identified for improvement Positioning of RMSA within Institute 2. Pre-Assessment Session 3. Assessment Workshop Facilitated session to establish current and desired future state – focus on priority areas established in step 2. Diagnostic results are qualitative and quantitative (based on voting) Awareness building and prioritization of topics for assessment – includes presentation, discussion, and completion of questionnaire Prioritized categories for assessment

10. Sample Definition Governance: Within our Institute, roles and responsibilities of management and professional staff have been clearly defined, communicated and are understood by those involved. Managers have authorities commensurate with their responsibilities, and are accountable for their actions. A comprehensive approach to governance is in place, including: • use of a defined and understood accountability framework; • use of an independent, expert advisory body; • documenting and communicating decisions in a transparent manner; and • having a fair and independent process to manage conflicts of interest.

11. The Assessment Scale • Discussion to focus on: 1) strengths (what is done well), 2) what can be done better (opportunities); and 3) some challenges/barriers to keep in mind; • The qualitative information is key – the numbers provide some scale of the gap that exists (“as is” and “to be”) to help in prioritizing issues.

12. Progress to Date and Moving Forward • Pilot testing of content and process • Results to date have been positive • Process has been well-received • Adjustments are expected to refine the content, and framework • Pilots to be completed by late Fall 2005 • Recommendations to be made to NRC senior executives regarding: • broader rollout of the tool based on pilot experiences • opportunities for corporate follow-up to address cross-cutting issues/gaps (e.g., training, corporate resources and support)

13. Some Messages to Date • Keep tool flexible rather than policy – cannot have a cookie cutter approach • Use of tool by institutes in two ways: • As quick checklist (Are we doing this or not?) • As a full assessment (How well are we doing in these various areas?) • RMSA should be positioned for use as an internal management tool rather than as a means of measuring institute performance and resource allocation • Accountability needs to be built in for follow-through on improvement • More support could be provided at the corporate level to institutes for research management – specifics will emerge with assessment activities.

14. Questions? For further information: Flavia Leung National Research Council of Canada Flavia.Leung@nrc-cnrc.gc.ca Website: www.nrc-cnrc.gc.ca