Excellence in Research

1. Excellence in Research Rachel Applegate, Assistant Vice Chancellor for Faculty Affairs IUPUI

2. Agenda 9:00 a.m. Welcome 9:10a.m. Overview of the Criteria to Demonstrate Excellence in Research 9:25a.m. Resources to Support Excellence in Research Simon Atkinson, Vice Chancellor for Research 9:45a.m. Metrics for Research Assessment Heather Coates, Digital Scholarship Librarian 10:00a.m. Flash Guide: eDossier and external-reviewer documentation for research 10:15 a.m. Panel: Jennifer Guiliano, History/School of Liberal Arts Amanda Friesen, Political Science/School of Liberal Arts Edward Berbari, Biomedical Engineering/Engineering and Technology, (current chair, campus Promotion and Tenure Committee) 10:45 a.m. Questions 11:00 a.m. Adjourn

3. Overview Criteria for Excellence in Research Here: Campus level! Consult department and school criteria

4. Faculty classifications and research Tenure-track: from assistant to associate + tenurealso satisfactory in teaching and service Tenure-track: from associate to full (with or without tenure)also satisfactory in teaching and service Research track: from assistant to associate research scientist from associate to full professor no teaching service as unit requiresscholar Tenure is not voluntary Promotion is voluntary

5. For research scientists/scholars Click here for full chart Chart for guidelines

6. For tenure track Click here for full chart Chart for guidelines

7. Building blocks of excellence INTEGRATION- Evidence of integration of all areas of endeavor appropriate for rank REFLECTION - Approach is reflective, systematic and purposeful QUALITY - Evidence of quality work and significant achievement FUNDING - Supports current and ongoing program of research

8. Characteristics of excellence FOCUS – Developing a body of focused work that extends or advances knowledge and brings recognition SCHOLARLY DISSEMINATION – Disseminating results through peer-reviewed publications, presentations or other media REPUTATION – Developing an emergent or sustained national reputation IMPACTFUL OUTCOMES – Documenting the influence or impact of one’s work

9. More about excellent research Evidence of dissemination of high quality scholarly work: peer-reviewed presentations at valued venues, publications in top tier journals Significant contributions to the knowledge base that improve or extend the work of others National/international recognition of expertise and the quality of the research Acquisition of external grant funding from competitive, valued sources Evidence of independent focused ongoing program of research Awards and recognition of research excellence

10. Independence Required at campus level for: Associate (and full) tenure track professor Full research scientist NOT required for: Associate research scientist Intellectual independence, original contributions; the work can be collaborative

11. Document your contributions to multi-author studies. P&T Guide: Candidate Statement, p. 16“Candidates engaged in interdisciplinary work or team science should make every effort to represent their contribution to collaborative scholarship clearly, as well as the significance and value of any interdisciplinary approach they are pursuing. Candidates should carefully document their individual contributions within this context and should also demonstrate some level of independent research beyond the team science work.” “Candidates should be careful to provide clear and sufficient information about their individual roles in collaborative projects, publications, presentation, or grants” More about independenceespecially regarding multi-author work

12. In the Research and Creative Activity section: p. 21“Increasingly, research or creative activity involves collaboration. Such collaboration across institutional and disciplinary lines is encouraged. Candidates must be careful to document the extent and form of their contributions to collaborative work. They should make clear their individual role (e.g., conception of work; acquisition, analysis, and interpretation of data; writing, revisions, and other communication; administrative and material support; corresponding or primary authorship) in such collective activity, preferably as related by colleagues involved in the joint work. Department or school/unit assessment of the individual contributions of the candidate who works with more than one author or collaborator must be included.” “As appropriate, the candidate should address achievement of independence from mentors and the establishment of an independent line of inquiry from prior mentors.” More about independence

13. “They should make clear their individual role (e.g., conception of work; acquisition, analysis, and interpretation of data; writing, revisions, and other communication; administrative and material support; corresponding or primary authorship) in such collective activity, preferably as related by colleagues involved in the joint work.” This can be a simple email exchanged, saved as a pdf: You to co-author: Dear XXX, on YYY project, my role was A and B and C. Could you please confirm this or let me know how else I should describe it? This confirmation is necessary for my promotion and tenure process. Use your own IU, IUPUI email. Have them use their own institutional email (no gmail!) and their professional signature block. More about obtaining confirmation of contributions to collaborative work

14. Key aspects: Especially for tenure-track, and for full research scientist Independence Impact Trajectory Peer review ……Dissemination outlets ……Grant reviews ……External letter writers Arrival! National reputation = full

15. Evidence fordocumenting research Chair You And chair You External You You You You And external You

16. Does prior work count??? YES and NO. For tenure: trajectory For full: national reputation “For faculty, publications and presentations in rank at another institution prior to appointment at IUPUI will be considered part of the candidate’s record. The overall pattern of productivity over time will be scrutinized, with emphasis place[d] on recent work and scholarly trajectory”(p. 29) “In many cases, it is understood that national reputation depends, in part, on foundational work that may have occurred earlier in the candidate’s career.” You build as you go AND You can’t stop in place

17. Brief guide: Acquire and sustain the resources to support research: start-up package, internal grants, external grants, school resources, study sites, collaborators. Do good research. Disseminate it in the best outlets: the highest possible return for your time and effort. Make sure your chair knows and agrees on the quality of your research and its publication venues. Keep track of your collaborators and your co-authors; document your individual contribution (confirmatory letters from co-authors are vital). E-dossier workshops: Oct. 29 and Nov. 20

18. People and responsibilities - within the PT cycle Candidate Chair Department committee School (unit) committee Statement CV Research examples Dossier Evaluation of outlets External letter solicitation Chair’s assessment Department standards: closest disciplinary examination School standards: broader, but still colleagues

19. Candidate Dean Campus P&T Committee EVC Kathy Johnson independent vote Chancellor Paydar and Pres. MacRobbie See next slide: Updating, rebutting negative votes People and responsibilities - within the PT cycle Update materials? Evaluation: own, plus summing up external, chair, department and school Comparison to campus standards Evaluation of process vote vote, submission to trustees

20. Once you “submit” your dossier you CANNOT change anything as originally submitted.You CAN put revised materials or additional materials in the Supplemental Materials folder. Label the files clearly with the type (e.g. CV) and date (e.g. Jan.15 2019). At every voting level you will be notified of the outcome. If you are a candidate for tenure AND the outcome is negative (the dean or chair vote NO, or there is a MAJORITY vote at the dept or school committee) you can request reconsideration. Do so in the Supplemental Materials folder; notify your chair; do so before the next level. Pg 31 of Guide. If you are a candidate for promotion only you cannot formally request reconsideration. You CAN upload supplemental materials and reviewers will be notified. Responding to the dossier process

21. New information: Grants have been approved (not just submitted) Articles/items have been approved (not just under review) Significant external awards or invitations have been received and confirmed Label clearly. Focus on major accomplishments. Do not over-use. Everybody goes “up” on five years of completed work, not the sixth year when the dossier is under review. New arguments: You believe that the chair, dean, or committee letters have inaccurate or misleading or incorrect information. Address these specifically and concretely. Your original candidate statement will remain. These should supplement it, not replace it. Use primarily when there are negative judgments on tenure. (Not just individual committee member “no” votes). Seek advice! Using the supplemental folder

23. Check with your school and department for your role in helping identify external reviewers. They may or may not ask you to provide a list of possible reviewers. You must provide a list of people who cannot act “at arms length” (p. 26 of P&T guidelines). Former or current mentors; definitely includes dissertation chair Co-authors or scholarly collaborators in the last five years. • Generally, this means co-authors of articles, and co-PIs. It may not include co-Is, if your role and theirs are parallel and not controlled by either. More about external letters/external reviewers

24. Co-authors more than 5 years in the past Speakers at conferences or invited to campus, co-panelists if invited by others, fellow committee members (if you are not the chair) They can know about you. They can have met you in scholarly circumstances. They can have followed your work. They do not need to know ALL your areas of research. If your work spans disciplines, someone from X discipline and another from Y discipline may be appropriate reviewers. They should be able to comment on the scholarly quality of your work: be able to recognize, evaluate, and describe it, even if they themselves do not do that exact work. Generally, these are okay for external reviewers:

25. Your chair will do this. Do not contact any reviewer knowingly, or about the tenure review process. If you are communicating with someone in your field about something else, go ahead but do not mention your tenure candidacy. Your chair can call upon other people in the School or at the campus level to help identify reviewers. It may take contacting up to 20-30 people to get people who are willing, able, and actually do produce a letter. This starts early: be prepared the year before. Soliciting external letters: what you need to know

26. IU Office of the Vice President for Research Simon AtkinsonVice Chancellor for Research, IUPUI Supporting your research

27. OVCR- research.iu.edu OFAPD - faculty.medicine.iu.edu Help from Campus • Workshops at OVCR (campus), OFAPD (IU School of Medicine Office for Facuty Affairs and Professional Development. Open to all IUPUI faculty) • Research Onboarding Orientation – October 17th • Finding Funding • Proposal Writing • Proposal Development Services • Mentoring – EMPOWER Program • Search for Funding Opportunities/Collaborators – Pivot Searches, Academic Analytics

28. What do you mean by “impact”? Heather Coates, University Library Metrics for Research Assessment

29. Core Concepts & Terminology Metrics are notdirect measures of quality or impact Metrics can be indicators of attention, engagement, use Context is crucial for reviewers to accurately interpret

30. Terminology Key Terminology • Metric – a measure for quantitatively assessing, controlling or selecting a person, process, event, or institution • Indicator – observable, measurable phenomena linked to research evaluation concepts; the link between indicator and concept may vary from weak to strong (Sugimoto & Lariviere, 2018) • Impact – includes scholarly, advancement of knowledge, clinical implementation, community benefit, economic benefit, and legislation and policy (Becker Model of Impact Assessment)

31. level type research metrics Article/item Journal as a whole Author Citations Web analytics Social media analytics context Discipline Purpose of work Audience

32. Levels of metrics Journal/Venue Level Metrics Output/Article Level Metrics Author Level Metrics

33. What kinds of metrics? • Citation activities person A cites person B’s item. Rough availability: Google Scholar • Web activities primarily “views” and “downloads” Rough availability: disciplinary repositories (e.g. arXiv), institutional repositories (like IUPUI scholarworks; your own pages) • Social media activities likes, follows, forwards/shares/re-

34. Journal-level metrics • Journal Impact Factor (Clarivate Analytics) • 2-year JIF • 5-year JIF • Eigenfactor Metrics • Eigenfactor Score (total journal performance) • Eigenfactor Article Influence Score (score per article; comparable to JIF) • Journal acceptance rate • Circulation/subscriptions • CiteScore Metrics (Elsevier)

35. Item-level Metrics All products • Citations • Raw counts • Web metrics • Unique visitors • Visitor location • Altmetrics • Tweets • Mendeley Readers • Blog mentions/citations • Wikipedia citations Journal Articles only • Normalized citation metrics • Relative Citation Ratio (NIH - https://icite.od.nih.gov/) • Field Weighted Citation Impact (Scopus) • Eigenfactor Article Influence Score

36. Author-level Metrics • h-index • i-10 index (Google Scholar)

37. Don’t forgetQualitative & other evidence • Awards, recognitions, honors • Invitations to speak about your work • Use of your scholarship in a course, as reflected on a syllabus • Use of your scholarship in professional practice • Use of your scholarship by policy-makers or legislators • Use of your scholarship by the community

39. Responsible use Candidates Reviewers Remember the purpose of the evaluation Recognize the limitations of the metric and evidence more generally Determine whether the metric or evidence supports the candidate’s case • Start with your story • Choose metrics and evidence to support your case • Choose your words carefully

40. Example

42. For scholar, produced by Univ Library librarians upon consultation

43. Scholar example, continued Interdisciplinary - Citations

44. Illustrating international usage

45. Support available IUPUI University Library Center for Digital Scholarship http://ulib.iupui.edu/digitalscholarship/impact Heather Coates, hcoates@iupui.edu Jere Odell, jdodell@iupui.edu Caitlin Pike, caiapike@iupui.edu http://www.metrics-toolkit.org/ On your own

46. Flash guide: eDossier and external reviewer documentation

47. You need to prepare materials outside the edossier For distribution to external reviewers: Candidate statement (7 pages) CV in IUPUI format / CV in disciplinary format Key research pieces often parked in Box Winter->Spring before the campus cycle starts

48. eDossier available for your use in mid-summer Candidate statement + CV Main dossier candidate sections (50 page limit, including candidate statement, not including CV) Reflection and description Appendices Full, original, and/or raw items: student evaluation reports, copies of articles, letters from collaborators Not used at the campus level. Not very used at the schools

49. SECRET NEWS You can use DMAI to generate some of your P&T CV

50. First-time users: Learn how to import your citations Current sources include: Google Scholar, ZoteroRefWorks, Web of Science via a BibTeX file Scopus, PubMed, CrossRef directly 2019: link to ORCID profile if you have one