1 / 34

August 2014

Research Grade Evaluation (RGE) and Equipment Development Grade Evaluation (EDGE) Panel Member Training. August 2014. Resources. Office of Science Quality and Integrity intranet site, http://internal.usgs.gov/quality_integrity/rge_edge/

ryder-watts
Télécharger la présentation

August 2014

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Research Grade Evaluation (RGE) and Equipment Development Grade Evaluation (EDGE) Panel Member Training August 2014

  2. Resources Office of Science Quality and Integrity intranet site, http://internal.usgs.gov/quality_integrity/rge_edge/ • RDSR portal (http://internal.usgs.gov/quality_integrity/rge_edge/RDSRportal.htm) - used to submit and access RDSR packages. • Office of Personnel Management’s (OPM’s) Research Grade Evaluation (RGE) and Equipment Development Grade Evaluation (EDGE) guides • USGS Research and Development Evaluation Process Handbook - how USGS implements the OPM guides • RGE, EDGE, and ST Checklists • Training materials RGE/EDGE coordinators: John Thompson and Judy Back, rge-edge@usgs.gov

  3. First-level RGE and EDGE panels • Rate the research/development contributions and impact of an employee using the RGEG/EDGEG and USGS checklists. • Evaluate the R&D and related work in terms of scientific quality and impact • Address career progress, balance of scientific activities, relevance of work to USGS goals, & content and quality of RDSR. • Identify significant work that is not R&D in nature. ID approximate % of time and impact on the position. Positions where R&D is less than 25% should not be rated by the panel. • Make final recommendations for all retentions in grade, conversions to RGE/EDGE, and promotions through GS-13. • Provide promotion recommendations and justifications to GS-14/15 and ST for the second-level panel • Review operational employees who request conversion to RGE/EDGE position • Provide constructive feedback to employees All evaluations, discussions, and documentationmust be kept confidential

  4. Research versus Development 4

  5. Four RGE/EDGE factors RGE EDGE Factor 1: Research Assignment Nature of the Assignment (scope, complexity, importance)(scope and complexity, objectives, means available, expected results) Factor 2: Supervisory Controls Supervision Received Factor 3: Guidelines and Originality Guidelines and Originality Factor 4: Contributions, Impact, and Qualifications and Contributions Stature (demonstrated ability, products, advisory activities) When scoring a package, each of the four factors is assigned a level, defined by OPM, with corresponding points. The sum of the scores you assign for the four factors determines the grade in which an employee is working.

  6. Science Impact Professional Society Services Societal Relevance Technical Assistance Core R&D Work Peer Reviewed Research Leadership and Direction Education and Outreach Strategic Planning Research and Development activities to consider

  7. 1st-level panel composition • Chair – Scientist who has served on previous panels. • Panel members – Panel members are research/development peers. The panel chair assigns members to be primary reviewers for specific scientists. • RGE/EDGE coordinator – Guides panel process and takes notes during the deliberations. • HR specialist – Attends panel as non-voting member to advise on procedural and interpretive issues.

  8. Roles and responsibilities – panel members Be prepared before the panel meets: • Thoroughly review the appropriate OPM RGE or EDGE (part 3) guide and the USGS RGE/EDGE/ST Factor Checklists.They define the key elements in each factor and aid in evaluation of each factor. • Thoroughly review the RDSR package for each employee. • Review and consider career experience, significant contributions, and bibliography • Read all publications listed under each scientist’s significant contributions (section 19) • Carefully consider evidence of impact and stature of the scientist • Score each package (for each of the four factors). • Use the package and the appropriate guide and checklist (RGE or EDGE) to determine a degree level and corresponding score for each the four factors. If you think a GS-15 employee exceeds the maximum scores, consult the ST checklist. • If an individual package does not fully meet a particular degree level, the next lower degree level must be assigned. • Take notes that indicate the rationale for each factor score. Record initial scores, notes, and questions or discuss during the meeting on the Panel Member Evaluation Score Form (exhibit A-4 for RGE and B-4 for EDGE).

  9. Roles and responsibilities – panel members (cont’d.) During the panel: • Be prepared, objective, and actively participate! • Confidentiality – nothing leaves the panel • All documentation, files, and notes are returned/destroyed after panel deliberations • Delete all digital files from your hard drives; destroy CDs/DVDs that contain panel information After the panel: • No outside discussions about panel deliberations or findings • Do not disclose identity of panel members

  10. Roles and responsibilities – primary reviewer Prior to the meeting: • As with all other employees being reviewed, thoroughly review the RDSR and PD, read the significant publications and score the individual. • Supplement the package with additional information – As a minimum, consult the supervisor; then, consult as many references as necessary to understand the body of work. You may use, but are not limited to, the references provided at the end of the RDSR. You may not consult with the individual being reviewed. If you run into problems, contact the RGE/EDGE coordinator. • Make certain you will be able to address each of the four factors and clarify scientist’s role on scientific teams/co-authored publications if it is not clearly indicated in RDSR. • For each scientist you will present: Provide a digital primary reviewer write up for each of the four factors to the RGE/EDGE coordinator before the panel begins. The panel will use this to help craft the feedback, so do not quote from scientist’s PD, RDSR, or written responses from references. All communications with references must be confidential.

  11. Roles and responsibilities – primary reviewer (cont’d.) Areas that may need clarification: • Does the scientist generate ideas/make interpretations independently? • What is the employee's role in cooperative efforts? Lead role or team member role? What were the scientist's contributions? • To what extent are existing technologies/methodologies applicable to the work? Has the employee moved science/technology forward? • What was the scientist's role in listed significant contributions, significance of results, and impact? Do the individual and (or) collective team contributions move the science/technology forward? Advance USGS strategic goals? Have societal impact?

  12. Roles and responsibilities – primary reviewer (cont’d.) Areas that may need clarification (cont’d): • How widely is the employee recognized and consulted in their field? Employee’s reputation among scientific and technical peers? How is leadership manifested in the employee’s record? • Have recent products been assessed by scientific/technical peers? Has sufficient time lapsed to assess the impact and relevance of findings? • Does the employee perform a full range of activities in the research/development cycle (from conceiving ideas through writing interpretive reports for technical and non-technical audiences)? Communicate with non-science groups or policymakers regarding nature, impact, and application of results to societal problems?

  13. Roles and responsibilities – primary reviewer (cont’d.) During the meeting: • Present information about the assigned employee’s career, current assignment, scope of responsibility, contributions, and impact. Clarify written materials for other panel members and provide information that is lacking in the written material but is needed to make a fair evaluation. • Do not advocate. Be objective in presenting and clarifying information for the panel.

  14. Roles and responsibilities – panel chair Before the panel meets: • Assign a primary reviewer for each scientist being evaluated. May consult with Science Center Directors, National Capabilities Mangers, RDs, ADs, RGE/EDGE coordinator and other appropriate personnel on these assignments. Remind primary reviewers of their responsibilities. Strongly encourage panel members to attend training. • Emphasize to all panel members the importance of conducting a thorough and objective review and rating of each scientist before the panel meets and the importance of maintaining confidentiality. Panel members should not rely on the primary reviewer as a primary source of information. • Ask primary reviewer to provide a descriptive write up for each of the four factors to the RGE/EDGE coordinator before the panel begins.

  15. Roles and responsibilities – panel chair (cont’d.) At the start of the panel – provide guidelines and emphasize: • Importance of objective, unbiased ratings using RGEG or EDGEG • The importance of providing constructive feedback to the employee • Importance of evaluating the quality and impact of publication and non-publication contributions when scoring factor 4 • How to evaluate contributions that are part of a scientific team • Importance of identifying when RGEG or EDGEG is not appropriate for evaluation. Note: research/development must be at least 25% of efforts to be rated by the panel

  16. Roles and responsibilities – panel chair (cont’d.) During the panel: • Ensure that each panel member’s scores are posted for each of the factors before beginning discussion of employee. • Ensure primary reviewer presents information to the panel for discussion. The panel is free to consult with anyone (except for the scientist being reviewed) who has additional information relevant to a case, in person or by phone during the panel. • Lead and manage discussion. Ensure that all panel members participate and that discussion stays on track.

  17. Roles and responsibilities – panel chair (cont’d.) During the panel (cont’d.): • Discuss all four factors to provide documentation for each consensus score. • Ensure that differences among scores are thoroughly discussed (especially >2 points for factors 1-3 and >4 points for factor 4). • Work towards getting consensus scores. Ensure that the final consensus score for each factor is justified in writing and that the justification correlates with the criteria for that score under the RGEG/EDGEG.

  18. Roles and responsibilities – panel chair (cont’d.) At the conclusion of the panel: • Ensure that the final summary comments on each Individual Panel Results form is approved by all panel members. • Complete and sign the Panel Recommendation Summary form. • Send final documentation to the appropriate RGE/EDGE coordinator, John Thompson or Judy Back

  19. Roles and responsibilities – RGE/EDGE coordinator Before the panel meets: • Ensure that all panel members have access to RDSR packages at least two weeks before the panel meets. • Provide training. Strongly encourage panel members to attend training. • Work with panel chair to determine date and location of panel. Start of panel – provide guidelines and emphasize: • Importance of confidentiality of materials and discussions – nothing is to be shared outside of the panel deliberations • The process that will be used to discuss and evaluate each employee and the importance of reaching consensus • It is never appropriate to discuss personal/performance issues about an employee during peer-review panel discussions

  20. Roles and responsibilities – RGE/EDGE coordinator (cont’d.) At the conclusion of the panel: • Re-emphasize the importance of confidentiality. Panel members must not disclose any information/panel deliberations. It is line management’s responsibility to provide feedback to the employee. • Send panel results (final Individual Panel Results forms for scientists not going forward to the second-level panel) to the Science Center Director/National Capability Manager. • Reiterate that all digital and non-digital documentation (RDSRs, notes, etc.) from all panel members, other than the final copies of the Individual Panel Results and Panel Recommendation Summary forms must be deleted or shredded.

  21. Special topics: Role of the position description • The current position description (PD), including a copy of the signed coversheet, is used by the panel to clarify the official work assignment (factor 1). • If a PD is inaccurate, the employee is responsible for drafting a statement that outlines the current work assigned by management and obtaining the supervisor’s signature to show concurrence. This should be part of the RDSR package. • Panel members should note discrepancies between the official PD of record and the assignment (section 11) and accomplishment (section 12) portions of the RDSR. Where appropriate, panel results should recommend that the PD be updated and should note the impact of discrepancies on the scientist’s career.

  22. Special topics: Mixed positions • Panels only evaluate the research/development portion of the position, but need to understand the scope of the whole position to give proper weight to the recency of the work. • Panels may have difficulty distinguishing between research/development and operational work. If necessary, • Discuss questions regarding the nature of the work with the supervisor. • Obtain position classification advice from the servicing personnel office.

  23. Special topics: Scientific Teams • When assessing Factor 1, consider the individual’s responsibility for research priorities, guidance, and productivity and credit the assignment in the following manner: • Assessment of Team Leaders/Project Chiefs should reflect the scope and character of the entire project being conducted by the team, provided the scientist actively guides the research being done by the entire project. If the Team Leader/Project Chief does not actively guide the research of the project, his/her role should be assessed in the same manner as a team member. • Assessment of a team member’s assignment should reflect the specific project, nature, scope, and difficulty of the assignment conducted by the individual and the contribution of that work to the impact of the larger project.

  24. Special topics: Assessments and Monitoring • Assessments and monitoring are mission critical for the USGS and can include a mix of operational as well as R&D components. • An employee’s R&D role in an assessment or monitoring project depends on the performance (originality, creativity, etc) of the person in the job. If a scientist is spending at least 25% of his/her time and effort conducting R&D, it is appropriate for the scientist to be evaluated by a peer-review panel. • Examples of assessment/monitoring-related research activities: • Development of new assessment/monitoring protocols and tools • Development of innovative decision support system tools • Use of assessment/monitoring data to develop/test hypotheses • Development of new predictive models • Innovative integration of interdisciplinary assessment /monitoring results • Discovery of previously unknown processes or principles

  25. Special topics: Technology Transfer • Synthesizes and translates scientific findings and/or results into a more usable format for decision makers and adds value to the science through additional interpretation, synthesis, modeling, or translation. • Should be directly related to and derived from official peer-reviewed R&D accomplishments and considered as evidence of the employee’s stature in the field. • Panel members should consider: • the impact of these activities on R&D in the field of expertise; • contribution to the resource and the customer; and • measurement of quality and quantity. • Examples of science application and transfer products: • Web-delivered analytical tools • Models • Maps and GIS products • Decision Support Tools • Reports (USGS series and other) to actions agencies (e.g., BLM, FWS, EPA and their state and local counterparts, NGOs and the private sector)

  26. Conversion scales for RGE and EDGE 26

  27. Where to find the information in the RDSRRGE 27

  28. Where to find the information in the RDSREDGE 28

  29. Final panel documentation – only two panel forms are preserved: (1) Individual Panel Results Form – for each scientist reviewed

  30. Individual Panel Results Form • Documents panel’s assessment in relation to four factors. Include final consensus score and justification (1-2 paragraphs/factor) for each factor. • Provide helpful and constructive feedback to employee. • Be accurate. Cite the scientist’s record and refer to the checklist. Substantiate panel findings. • Do not quote from the scientist’s RDSR, position description, or others you have consulted for the review. • Forwarded to the second-level panel when reviewing recommendations for promotions to GS-14 and above and below-grade findings.

  31. Individual Panel Results Form, cont’d. • Factor 1: Assignment • What research/development is the scientist currently doing and what is the scope of the work? • How complex are the problems being addressed? • What is the importance of results/expected results? • Factor 2: Supervision • How is the work assigned? • How much guidance is given? • What level of responsibility does the scientist have? • How are the results reviewed?

  32. Individual Panel Results Form, cont’d. • Factor 3: Guidelines and Originality • What guidelines (literature, procedures, instructions, precedents) are available for the current work? • What is the level of originality that the scientist brings to the work? • Factor 4: Contributions, Impact, and Stature (RGE) Qualifications and Contributions (EDGE) • What are the scientists publication and non-publication contributions? • What impact have the scientist’s contributions had? Include evidence. • What is the stature of the scientist in his/her field? In the broader scientific community? Include evidence.

  33. Final panel documentation – only two panel documents are preserved: (2) Panel Recommendation Summary Form – for each panel

  34. Resources: Office of Science Quality and Integrity intranet site, http://internal.usgs.gov/quality_integrity/rge_edge/ • RDSR portal (http://internal.usgs.gov/quality_integrity/rge_edge/RDSRportal.htm) - used to submit RDSR packages for review. • Office of Personnel Management’s (OPM’s) Research Grade Evaluation (RGE) and Equipment Development Grade Evaluation (EDGE) guides • USGS Research and Development Evaluation Process Handbook -how USGS implements the OPM guides • RGE, EDGE, and ST Checklists • Training materials RGE/EDGE coordinators: Judy Back and John Thompson, rge-edge@usgs.gov

More Related