Quantitative Assessment of Occupational Skills Shortages: the impossible challenge?

Quantitative Assessment of Occupational Skills Shortages: the impossible challenge? Eur Ing Dr. Matthew Dixon Semta Fellow, SKOPE/ SKOPE Research Fellow, SEMTA

The Sector Skills Council for Science, Engineering and Manufacturing Technologies MD ‘baggage’ • Control Engineering (Dynamical systems) PhD* • Long interest in the relationship between Science (Analysis) and Policy • (obsessive?) desire to find out what a Skill Shortage is… (and how you could prove you really had one) • Real concern about the gap between skills theory (labour economics) and (policy) practice * no sign of an economics degree…

The Sector Skills Council for Science, Engineering and Manufacturing Technologies Skill Shortage(s): • Sounds plausible; many claims, but they somehow mostly seem to slip away under scrutiny… (all sectors need to claim shortages(!), and skills policy wonks need ‘em…); • Long been a holy grail - labour economists have apparently been sceptical; • Skill shortage could be viewed as central to skills policy – the most obvious ‘market failure’ for policy to address??; • UK policy has considered Skill Gaps (internal to an employer) and Skill Shortages (external) – Gaps* are employers’ responsibility? Shortages are presumably not… so that makes them the State’s responsibility?; • UK skills policy has generally worked to increase supply in areas where shortage has been assumed/accepted, but public investment doesn’t always result in supply increasing in precisely the area intended… * but skill shortages can(?) turn into skill gaps…?

The Sector Skills Council for Science, Engineering and Manufacturing Technologies Workforce Structure • The sectoral emphasis for policy (institutions) is understandable, but arguably flawed – and allows (increases?) confusion • Sectoral bodies naturally ‘think sectors’, bringing assessments like “x% of employers in the sector experience recruitment difficulties”…(??) • The employer emphasis has led to concentration on reported recruitment problems (employer surveys on HtFVs -> SSVs) • While essentially subjective, this evidence allows comparative statements… • However the labour markets (trading skills through recruitment) are essentially occupational – the Matrix!

The Sector Skills Council for Science, Engineering and Manufacturing Technologies Assessing Shortage for Managed Migration policy (1): The Sector Advisory Panels (‘SAP’s): • “Skills 99” , the ‘Stevens Report’, 98/99 extremes & the DTI working party • Several SAPs established in the early ‘noughties’ (MD served on ‘ITCE’ and Engineering panels) • From anecdotal evidence to the ITCE Panel methodology – available data sources and IT Practitioner skills characteristics • Many practical implementation challenges..() • Presence of both sides produced informed assessment of specific claims… • Assessment of State of ‘national occupational labour markets’ vs. the Resident Labour Market Test (RLMT) • Experience and MD’s conclusions

The Sector Skills Council for Science, Engineering and Manufacturing Technologies Assessing Shortage for Managed Migration policy (2): The arrival of the MAC: • Occupations from the start • First practical comprehensive use of indicators that labour economics talks about (not least, price!) • Data (from general economic databases) largely based on the Standard Occupational Classification (SOC) • Recognition that top-down data needed complementing with bottom-up evidence • MD’s pleasure!

The Sector Skills Council for Science, Engineering and Manufacturing Technologies The challenge: Using different sets of imperfect quantitative data for decision-making in one of the most contentious policy areas there is…

The Sector Skills Council for Science, Engineering and Manufacturing Technologies The (main) Issues: • Thresholds • Occupational Classifications • Sampling Frequency • The ‘Dovetailing’ • Implementation practicalities

The Sector Skills Council for Science, Engineering and Manufacturing Technologies Thresholds: • continuous (‘analogue’) data must be turned into a binary (‘digital’) outcome (shortage/no shortage) • ITCE Panel used the “horror years” as the ‘calibration’ reference • MAC uses movements in the available time series (top quartile?, top decile?, top centile?, top ‘x-ile’?) • So thresholds always arbitrary…(significant problem for this context…) • But using arbitrary thresholds on objective proxy indicators is (?) better than anecdotal evidence as a basis for ‘heavy’ policy decisions?

The Sector Skills Council for Science, Engineering and Manufacturing Technologies Occupational Classifications: • The big challenge…* • SOC, revisions, and ISCO (353 @ 4-digit categories not really designed for this heavy load!... > 26,000 ‘Job Titles’) • Occupations, roles and functions • Structure and granularity • MAC shortage list now almost entirely subsets of SOC 4-digit categories (i.e. SOC has been found to be simply not fine-grained enough to capture ‘real’ occupations in which skills are traded) • Semta now exploring ‘5-’ and ‘6-digit’ categories for engineering occupations • Would policy ever care enough to realise the importance of occupational information, and find the money for significant increases in survey sample and frequency…?? * also relevant for skills-, competence- and qualification- frameworks

The Sector Skills Council for Science, Engineering and Manufacturing Technologies Sampling Frequency: • Bi-ennial? Annual? Quarterly? • Trade-off between adequately capturing the labour market movements and sufficient stability for a ‘steady’ policy • Limitations largely a cost issue

The Sector Skills Council for Science, Engineering and Manufacturing Technologies The ‘Dovetailing’: • Top down and bottom-up evidence both have weaknesses • MAC has worked hard to listen to employers • Assessing individual employer claims really needs peer (‘sectoral’) review • Can the dovetailing ever be 100% meaningful? (e.g. SOC 5223…)

The Sector Skills Council for Science, Engineering and Manufacturing Technologies Implementation Practicalities • The ‘Points-Based System’ (PBS) processes (decentralised – i.e. ‘in country’ c.f. previous approach) • Ability of case workers to assess technical specifics? • Meaning of the Resident Labour Market Test (RLMT) in a world of recruitment by Web…? • Could the RLMT ever be adequately robust?

The Sector Skills Council for Science, Engineering and Manufacturing Technologies Valid/Robust assessment of occupational skill shortages: can it be done? • Kind-of, but is ‘kind-of’ dangerous? – limits to the use of analysis for policy? • The MAC’s work has begun to seriously address the gaps between (quantitative) theory and practice • The MAC’s appearance has triggered (much) greater interest in occupations within Skills policy (=>UKCES => SSC contracts) • How could we assess the limits of relevance of national analysis of occupational labour markets to the real experience of an employer?

The Sector Skills Council for Science, Engineering and Manufacturing Technologies What have we learned? • That skill shortage is still very difficult indeed to meaningfully assess • That the investment in the MAC is helping understanding (inc. opportunities for real testing of some theoretical models) • That Skills Policy probably needs to think harder about market failure, and what the State’s role is • That occupations are more important (for detailed labour market analysis) than sectors • That occupational classifications are worth improving (and investing in, if there is any money…) • Migrant labour as a source of supply is being considered more seriously than before, and there is a will to link migration and skills policy more strongly

Quantitative Assessment of Occupational Skills Shortages: the impossible challenge?