Broad-Based Office Skills Testing

1. Broad-Based Office Skills Testing Los Angeles County Office of Education PERSONNEL COMMISSION

2. The Problem • Many clerical/secretarial classifications • Differ on specific job content and levels of responsibility • Many obstacles to class consolidation - logistical, political, fiscal • Each class has a unique examination • Variably content and quality • Not consistently updated • 2-3 month recruitment & examination cycle time • No overall competency architecture across the classes • Core competencies and competency progression • Pass point paradoxes (lower pass points for higher level jobs based on selection ratios)

3. Purpose and Objectives • Improve testing quality, reliability, and efficiency • Improve coherence of clerical examination plans • Reduce redundant testing • Reduce time to place candidates on eligibility lists • Ensure eligibility lists are always available

4. Strategy • Core test battery for as many classifications as possible • Annual calendar of assessment opportunities (e.g., quarterly) • Computer-based testing • Score banking and certification to different exam plans • Different weights and cut-off scores for different classifications based upon relevancy and level needed.

5. Strategy: Similar to SAT & ACT for applying to college • Sign up and take the test • Apply to the colleges of choice • Send scores • Colleges set their own pass-points and weights for final selection decisions Sign up and take the test Apply to the jobs of choice Scores populated Each job has its own pass-points and weights for final selection decisions

6. Score Data Flow

7. Development of the Test Battery • Reviewed job analyses from Agency-Wide Classification Study - 2004-2007 • Re-stated duties into “generalized work activity” (GWA) statements with no reference to specific job content. • Map GWAs to each classification • Identify relevant assessment tools for GWAs • Via GWAs, map assessment tools to Classifications • Derive relevancy and weight of assessment tools to classifications.

8. Example GWAs • Greet, interact, and provide information to others in person and on the telephone • Verify, input, and correct data and information • Receive, sort, amd categorize information and materials • Understand, interpret, explain, and follow directions, rules, and regulations • Operate a personal computer and other common electronic office equipment • Format, edit, keyboard and type documents. • Gather compile, collate, and assemble information and materials • Maintain and coordinate calendars and schedule meetings and events • Receive, track, and record monetary/numerical information and data

9. Matrix of GWA with Classifications

10. Competencies Assessed - Tests • Following Instructions (MC) • English Usage & Grammar (MC) • Microsoft Outlook (MC) • Microsoft Excel (Performance, OPAC) • Microsoft Word Skills (Performance, OPAC) • Telephone - Customer Service (SJ, OPAC) • Sequencing and ordering speed and accuracy (timed) • Computational speed and accuracy (timed) • Checking and comparing speed and accuracy (timed) • Data Entry speed and accuracy (OPAC)

11. Matrix of Tests with GWAs

12. Matrix of Tests with Classifications

13. Result • Linkage of tests-to-tasks-to-classifications to establish job relatedness • Derived weighting scheme for each classification • Relevant tests • Relevant weights • Job specific competencies identified in the job analyses are assessed in specialized examination parts • Min Quals • Interview • Performance

14. Clerk Intermediate Clerk Senior Clerk School Clerk Senior School Clerk Data Control Clerk Senior Data Control Clerk Typist Clerk Intermediate Typist Clerk Temporary Office Worker Senior Typist Clerk Department Assistant, Dance Department Assistant, Music Department Assistant, Theater Department Assistant, Visual Arts Receptionist Reader Information Resources Specialist Media Dispatching Clerk Secretary Division Secretary Legal Secretary School Administrative Secretary Senior Division Secretary Executive Legal Secretary Executive Assistant Applicable Classifications

15. Affected Positions

16. Administrative Practices • Score Life: • 2 years • Test Administration: • Every 3 months • Test Retake: • Once every 6 months • Retake Scope: • Whole Battery – not individual tests for logistical reasons • Weighted or Qualifying-Only: • Qualifying Pass/Fail-only for Promotional Exams • Weighted for Open recruitment

17. Administrative Practices • Effective Scores: • Only the candidate’s most recent score, not the highest score, can be certified to a list. • For candidates whose scores enable them to be placed on an eligibility list (pass the exam), the certified scores stands for the duration of eligibility on the respective list. • Scores already certified are not updated by retakes • One’s score (and relative ranking) on a list can not be improved until eligibility expires (one year). • Multiple (10) Pass Points • Low individual test pass points • Reduced Adverse Impact • 3 apparent factors: • Computer skills, speed and accuracy, verbal

18. Recruitment • Continuous offering of the BBC • Continuous recruitment for all affected classifications • Establishes a pre-tested pool of potential candidates for the affected classes • For most classifications, limited or no public posting for job specific recruitments • Notifications sent to pre-tested pool, based on test cut-off score • Invitations to apply to take the specialized exam parts for the classification

19. No need to go to the general population for each recruitment

20. Status • Officially launched on February 9th, 2009 • Advertised on our PC website & Weekly Job Opportunities emails. • Two exam administrations in March & April phased in the BBC in the context of specific classification recruitments. • June and October 2009 BBC administrations as part of annual calendar

21. Process Improvements Since Start • Candidates are automatically notified via email when particular job classifications are open for recruitment. • Candidates receive their score report along with an Examination Information Sheet to address frequently asked questions. • Scores can (soon) be viewed on-line with ID and Password • Candidates who miss the Scantron exam portion can re-register to take the next exam administration. • Candidates who are unable to attend the first Computer Exam can attend the Computer Make-up exam typically scheduled the following week.

22. Benefits Received and Expected • Test quality improvement through • Data monitoring and psychometric analysis • Empirical validation research • Development of alternate forms • Administration efficiencies through • Managed efficiencies in test administration and score managements • Minimizing retesting of the same candidates for different classifications • Improved communications with candidates • Recruitment enhancement through • Maintaining a pre-tested pool of applicants • Having something to offer at job fares • Reduced eligibility list production cycle time

23. Next Project: BBA • Broad-Based Administrative Classifications • Administrative, Analyst, and Professional Series • Focus on college grads • Same general model • Launch in Q1, 2010

24. Thank you! Questions? Comments?