Publishing Productivity of US Academic Scientists: an Empirical Examination through Data Envelopment Analysis

1. Publishing Productivity of US Academic Scientists: an Empirical Examination through Data Envelopment Analysis Youngsun Baek Research Value Mapping Program School of Public policy Georgia Institute of Technology

2. Contents • Introduction • Background • Difficulties with Performance Indicators in the field of higher education • Efficiency and Data Envelopment Analysis (DEA) • Research Design: Publication producer model • Results • Most Productive Scientists (Good Apples) • Decomposition of Technical Efficiency (TE) into Pure Technical Efficiency (PTE) and Scale Efficiency (SE) • Returns to Scale (RTS) of each scientist • Disciplinary Efficiency (DE) and Individual Ability (IA) • 2nd stage regression (Tobit Model) • Disentanglement of TE into DE and IA • Conclusion

3. Introduction:Which scientist is more productive? Scientist A Scientist B Outputs • 10 articles/year, 2 books/year • 3 articles/year, 1 book/year • Research funding: $ 1,000,000 • # of coworkers: 30 • No teaching load • Research funding: $ 50,000 • # of coworkers: 5 • 3 classes to teach Inputs

4. Difficulties with performance indicators in the field of higher education “… In particular, appropriate adjustment needs to be made in order to prevent confusion between output and efficiency. For example , crude measureof research output can be obtained by a straightforward publications or citations count; but, this makes no allowance for the vast differences in resources…”, Johnes & Johnes (1995)

5. Difficulties with performance indicators in the field of higher education • Johnes & Johnes (1995) • Confusion between output and efficiency • Multiple inputs and multiple outputs • Difficulty with citation analysis • Adding together different types of publications • Hanney & Kogan (1991) • Problem with impact factors • Avkiran (2001) & Korhonen (2001) • Absence of market mechanism in the academic field. ⇒ Data Envelopment Analysis (DEA)

6. Efficiency and Data Envelopment Analysis (DEA) X1 Technical efficiency of A = OA’/OA A B G H C A’ F D E O X2

7. Methodological Strengths of DEA as a Performance Indicator • DEA can handle multiple inputs and multiple outputs. • It does not require an assumption of a functional form. • Producers are directly compared against a peer or combination of peers. • Inputs and outputs can have different units.

8. DEA Application to Performance Evaluation • DEA applications to evaluating academic organizations • Johnes & Johens (1995) ; U.K. university departments of economics • Korhonen (2001) ; Universities and research centers • Thursby & Kemp (2002) ; University licensing activities • Caballero et al. (2003) ; Allocation and management of university financial resources • Few studies conducted to evaluate individual researchers through DEA. ⇒ Focus on each individual scientist (Human Resource Analysis)

9. Multi inputs (Production Factors) Producer (Each Academic Scientist) Multi outputs (End Products) NNFC NNA DSM CM NNGSC YGPD NNB NAG USCIT NNSP G Contextual factors Publication producer model * NNFC: Normalized number of faculty coworkers, NNGSC: Normalized number of graduate students coworkers, NAG: Normalized amount of grants, NNSP: Normalized number of submitted proposals, NNA: Normalized number of published articles, NNB: Normalized number of published books, DSM: Disciplinary society membership, CM: Career mobility, YGPD: Year got a ph D. degree, USCIT: US Citizenship, G: Gender

10. Decomposing Technical Efficiency & Nature of Returns to Scale PTE BCC model 0.56 TE CCR model 0.45 SE 0.78 Output DRS 22% CRS 10% 68% IRS Input

11. DRS IRS Efficiency in each discipline Total Technical Efficiency Pure Technical Efficiency (Managerial Efficiency) Scale efficiency

12. Most Productive Scientists (Good Apples) • 9 scientists out of 109 were selected as most efficient scientists • O.28 faculty coworkers/ year • O.33 graduate student coworkers/ year • 1.15 proposals/ year • $160,277(present value-2000)/ year • 3.6 articles/ year • 0.6 book / year • 7 scientists out of 9 were male • Two thirds of them had US citizenship • 6 disciplinary society memberships • 10 career changes • 51 years old

13. The 2nd stage regression(Tobit Regression Model) • Dependent variable- Pure technical efficiency (DEA efficiency score) • Independent Variables - DSM: a variable for disciplinary society membership, CM: a variable for career mobility, Ph D in 60s, ph D in 70s, and ph D in 80s: dummy variables for year got a ph D degree, USCIT: a dummy variable for citizenship, G: a variable for gender) • ** At the 1% significant level • R-square 0.383921 • Adjusted R-square 0.334136

14. Disentanglement of disciplinary efficiency Source: Thanassoulis (2001)

15. Disentanglement of disciplinary efficiency Inter-Group Efficiency Comparison

16. Disentanglement of disciplinary efficiency Decomposition of total efficiency into Individual and Group Efficiency

17. Disentanglement of disciplinary efficiency Differential Efficiency Distribution

18. Conclusion • DEA can suggest an alternative performance indicator of publishing productivity. • Contextual factors • Existence of vintage effect in publishing productivity • No correlation between publishing productivity and disciplinary society membership • Negative relationship between career mobility and productivity • In order to catch up with the best performing scientists, improving publishing process is better way than trying to increase input levels such as coworkers and research grant. • Project management and effective collaboration strategies are crucial to increase publishing productivity. • Performance of each individual depends not only on individual ability but also on the characteristics of the discipline under which the person is working.

19. Thank you!