Quantitative Measurement of Project Manager Competencies

1. Quantitative Measurement of Project Manager Competencies Research Team 306 Implementation Session

2. Research Team 306Quantitative Assessment of Project Manager Competencies Academic Team Members Awad S. Hanna,University of Wisconsin- Madison Jeffrey S. Russell, University of Wisconsin- Madison Carol C. Menassa, University of Michigan  Alexandra E. Beletic, University of Wisconsin- Madison Industry Team Members Chair: Richard D. Bradford, Bechtel Group, Inc Vice Chair: Jeffery L. Moore, Ameren Missouri Dale C. Allen, Shell Global Solutions Robert Bryant, BP Refining & Logistics Technology Jose A Buitrago, Jr, CSA Group Richard D. Carrithers, Jr. , DTE Energy George Carter, Anheuser-Busch InBev Jeff Drinkard, Hargrove Engineers + Constructors Ian D. Etzkin, URS Corporation RoqueFrancisquez, SABIC - Saudi Basic Industries Corporation Bryan Hoerner, Audubon Engineering Company, LP Richard A. Krout, Walbridge Rita M. Norat, Air Products and Chemicals, Inc. Nancy Skinkle, Architect of the Capitol Frank Williamson, Wood Group Mustang

3. Agenda • Research Summary • Literature Review • Data Collection and Analysis • Tool Development and Validation • Tool Walk Through • Panel Discussion • Questions/Answers

4. Research Objectives The primary purpose of RT306’s research was to develop a consistent methodology to define and quantitatively measure the competencies of Project Managers (PMs) in order to: • Identify PM competency gaps • Focus PM training needs and development • Develop and validate an implementable and useable Project Manager Competency Assessment Tool (PM CAT)

5. PM CAT (Project Manager Competency Assessment Tool) • Provides a thorough list of competencies and definitions that a PM should possess to successfully lead, manage and execute a project • Assess the competencies of a PM and compares him/her to the industry • Improves PM effectiveness by targeting specific development needs to enhance their competencies in a personalized and more efficient manner • Enhance the selection process of qualified or potential PMs A user-friendly tool designed to quantitatively assess and score PMs’ competencies.

6. Background • PMs must have the technical and leadership expertise, training, and characteristics to bring about a successful realization of project goals • Should continuously improve their skills through development and training • Historically been evaluated by qualitative means • CII commissioned RT-281 – Project Management Skills for the Future in 2010 • Anticipate the PM competencies to respond to industry changes and future challenges • Served a s a platform for RT-306’s research

7. Research Methodology

8. Literature Review • Industry-wide need for a simple and quantitative PM Competency Assessment Tool • Existing PM competency models • Discover, include, define and properly weight all critical PM competencies • Organized into four competency groups: • Knowledge/Experience (altered from RT-281’s Technical/Virtual), • Management • Cognitive (& Attributes) • Leadership

9. Existing Competency Model examples…. NASA Development Framework Leadership Model: Management (NASA 2008) Corporate Education Group Project Management Competency Model (CEG 2008)

10. RT-306List of PM Competencies

11. Data Collection: Two Surveys Preliminary Survey • Objective of survey: • Eliminate unimportant competencies • Assign competency weights • Administered to RT-306 Members • 5-point scale Detailed Survey • Objective of survey: • Test competencies on industry PMs • Ensure each competency is statistically significant • Determine industry performance levels and benchmarks • Administered to PM Supervisors • asked to answer about an average PM and a “stellar” excellent PM • Questions were developed with the team and professionally edited to pinpoint the meaning of each competency

12. Competency Weights(from first survey)

13. Data CollectedRespondent and Company Information • 77 PM Supervisors from (at minimum) 29 companies • 154 PMs were analyzed

14. Company Locations: • United States (92%) • International (8%)

15. Excellent versus Average Project Manager

16. Data Analysis • Paired t-test • Tested if excellent PM scores were significantly different then the average PM scores • All competencies (except for one*) were statistically different and therefore necessary to quantifiably measure a successful PM • Used in the PM CAT • PM scores were calculated with weights from first survey • to make the industry distribution • to create benchmarks for the PM CAT • Formula to calculate PM scores: • where j denotes Project Manger • wi is the weight of the category • sij is the composite score of competency j for category i (this is the individual question score times the competency weight) *The one exception, where there was no statistical difference between average and excellent PMs, was the attribute “Credible/Honesty/Ethics/Integrity/Trustworthy/ Loyalty/ Empathetic”.

17. Formula Further Broken Down Scores were normalized at each step! 1. Overall Formula 2. Summed Categories 3. Summed Competencies within the Category (only Knowledge and Experience shown)

18. Tool Description • Reliably and consistently measure PM competencies • Excel 2007 (v 12) or later versions • Rates PM’s competence on an industry-based scale • Identifies training and development opportunities (competency gaps) • Helps in the selection or placement of PMs • Should be completed by the PM's Supervisor • Questions are job specific and relate to any phase of EPC projects • PM CAT can be administered at any time during PM’s career • can be administered multiple times on one person over time to see progression • Should not be confused with or substituted for a performance rating system

19. Tool Validation: Beta Testing • 14 PM supervisor participants • 93% accuracy • tool is suitable for use • of the limited overseas sample (3 respondents): 100% accuracy • Respondents said • the tool overall was useful to somewhat useful • 2.7 on a scale of 5 • user friendly to very user friendly • 3.35 on a scale of 5 • Asked if the research objectives of the tool were met: identity competency gaps, focus training and development, and develop and validate implementable/useable tool. • Yes: 10 • Maybe: 4 • No: 0 • Asked if they would use the tool in their company • Yes: 9 • Maybe: 2 • No: 3 • Tool enhancements done based on feedback received

20. The deployment of the PM CAT will help enhance overall project performance by improving the process by which Project Managers are developed and selected. This, in turn, will contribute to CII’s mission to improve overall industry performance

21. Tool Input • Questions focus on the competencies that a PM should possess to successfully lead, manage & execute a project

22. Tool Output • PM Competence Percentile • Rates PM’s competence on an industry-based scale • Illustrated on a graph against industry benchmarks • Detailed Competency Report • Highlights PM’s strong and weak competencies • Opportunities for development and training (competency gaps)

23. Percentile Score and Graph

24. Detailed Competency Results

25. Tool Walk Through…

26. Panel Discussion Research Objectives The primary purpose of RT306’s research was to develop a consistent methodology to define and quantitatively measure the competencies of PMs in order to: • Identify PM competency gaps • Focus PM selection, training needs and development • Develop and validate an implementable and useable PM CAT

27. Question and Answers