Training Results You Can Measure: Why Conducting Secretarial Skills Assessments Is Worth Your Effort

1. Training Results You Can Measure:Why Conducting Secretarial Skills Assessments Is Worth Your Effort March 13th, 2009 Corby GuentherSenior Technology Training and Development ManagerBingham McCutchen LLP corby.guenther@bingham.comBay Area Legal Trainers Organization http://groups.google.com/group/baltorg

2. Training Results You Can Measure

3. Things I learned and heard during a secretarial skills assessment project Why Are You Doing This To Us?

4. Some quotes from secretaries: • No, honestly, how is this going to affect my job/bonus/review? • We’ll form a union, and then you’ll be sorry! • We’re friends, you can give me a copy of the test! • I just want you to know that we hold you responsible for anyone who jumps off the bridge

5. Skills assessment really is a worthwhile project • Assessments help you take some of the “gooshiness” out of training • Bingham was forced to come up with standards for document production for the first time • Secretaries have taken more responsibility for their training • We now have hard data that we can use to design our training programs • We have been able to show our follow-up training is effective

6. Score Improvement Post-Training

7. But it’s far and away the hardest project I’ve ever had in this job • You have to get your firm’s administration interested in the idea of skills assessment • You need to make sure that the right group has ownership of the project • The skills you want to measure need to be decided on by a group that is fair and credible • The firm has to have a communication plan that speaks to the secretaries honestly and with one voice • You must be prepared to deal with the emotional fallout

8. A brief(ish) history of our long project • Bingham McCutchen has approximately 1000 lawyers and has been formed through a series of mergers • Secretaries in various offices had widely different levels of experience with our document production tools (Office, MacPac, iManage) • Our goal was to standardize document production across our diverse offices (11 US offices and London) • Our New York office had previously done an in-house assessment

9. A brief(ish) history of our long project • We started talking about using an outside resource to conduct a firmwide assessment more than four years ago • It took us almost a year to get administration on board and the project off the ground • We conducted two rounds of training and delivered a second round of assessments • Did anyone lose their job?

10. A brief(ish) history of our long project • We hired Theresa Lundquist (then at Perfect Access/Speer now Profisciencē Partners) to help us with this project • We chose to focus on Word and MacPac for this assessment (no Excel or PowerPoint) • Theresa helped us put together a list of “Core Competencies” we expected of our secretaries • Our NY document production department participated up to this point, but much of the work fell on trainers because of our “technical” expertise • Our assessment required secretaries to edit and format an actual Word document based on written instructions (not a simulation)

11. A brief(ish) history of our long project • D-Oh! While we were in the planning stages we announced a merger and needed to roll out Office 2003 • Assessments were put on hold and we developed Office 2003 training that also incorporated all our core competencies • Our plan to conduct assessments wasn’t announced to the secretaries until the week before training started in the first office • We had several meetings to talk to the secretaries about what the assessments would be like • Our timing wasn’t ideal

12. A brief(ish) history of our long project • Theresa Lundquist and I worked to create the actual document we would use in the assessment so it included all the skills we wanted to measure • We decided that we wanted to use one document across the firm for consistency– we chose to use a business agreement • Our trainers took the assessment as a pilot and we revised it accordingly • We started the “live” rollout in our small Walnut Creek office

13. Timing • Secretaries were given Office 2003 training office by office • We conducted assessments about four weeks after the secretaries were trained • In the meantime we offered refreshers and drop in sessions • We also gave the assessment to Help Desk staff • Eventually we were training, rolling out and assessing in several offices at once • Secretaries were told to expect their results in about three weeks

14. Logistics • Assessments were held in the controlled environment of our training rooms • Trainers proctored, sometimes with help from HR • Secretaries were given a set of instructions, a partially formatted document, and an example of how their final product should look • We limited the assessment time to 90 minutes • Trainers were there to answer questions about the instructions, check computers afterward. • In some offices we tried to have an HR person around in case emotions got out of hand

15. After the First Assessment • Theresa evaluated assessments emailed to her by the secretaries • She sent us back detailed results for individuals, offices and the firm overall • We chose not to give the secretaries scores, just put them in courses they needed to attend • Tasks were divided into seven categories • We set a “passing” threshold at 85% for each category, so secretaries had to take between zero and 7 classes • Theresa and I met with HR people from each office to explain and discuss the results

16. After the First Assessment • HR people handed out confidential results by hand and let secretaries know they were available to discuss them if desired • I took Theresa’s results and imported them into UniversitySite as learning plans • We scheduled follow-up training for each office • Secretaries could go into UniversitySite, see their learning plans and sign up for class • Anyone who receive more than 85% in every category did not have to take the second assessment • We could track the progress of individuals and the firm overall

17. Learning Plans in UniversitySite

18. Learning Plan Graduates

19. Status of “Learners” in UniversitySite

20. User’s View of a Learning Plan

21. Signing Up for a Class

22. The Second Round • Secretaries completed Learning Plans • We revised “refresher” training to make it more focused • Trainers taught with a document similar to assessment • There were lots of “Ah-ha!” moments where you could see people finally understanding a concept • Assessment instructions were simplified for second round • Attitudes were much improved the second time

23. Second Round Results

24. Second Round Results- Individual Tasks

25. What’s Next • No further plans for Word assessments • We did not require new learning plans for secretaries • We need to come up with a plan for assessing new hires and people brought in through mergers • We’re considering what applications we might do next • Our next assessment will probably be a simulation • Hopefully the secretaries will get used to the idea of needing to keep their skills up to date • ...and we’ll get used to the idea of using assessments to evaluate our training programs

26. Recommendations • Having an outside consultant is a huge advantage so you don’t have to reinvent the wheel • Decide what kind of assessment you want to do early in the process-- budget time and money accordingly • Make sure the project has a strong and actively involved sponsor • Involve other groups at each stage of the project • Secretarial representatives in defining core competencies • Local office representatives so they understand the entire process and have a stake in it • Keep attorneys informed about what you’re doing and why you’re doing it • Be clear about why you’re doing assessments

27. Recommendations • Do whatever you can to overcome differences between offices and between other groups • Don’t link your assessments to a new rollout unless that’s specifically what you want to assess • Think hard about how you want to assess people who perform different functions • Admin secretaries vs. document producers • Litigation vs. Business secretaries • Help Desk vs. “Real” users • Do you want to use one document type for everyone? • Don’t let this be a “secret” project. It just makes secretaries suspicious of your intentions • Try to set a casual, everyday tone whenever possible

28. Don’t shoot the messenger…

29. Recommendations • Understand that– despite your best intentions– much of your communication effort will be lost because people won’t hear or believe what you tell them • Make it clear that you’re assessing skills that you expect everyone to possess • Keep the instructions for the assessment as simple as possible • Use language everyone will understand • Get as much input from others as possible • Make the space you use for the assessment as comfortable as you can and plan for complications

30. Recommendations • Keep assessment documents out of the hands of secretaries. Someone will try to get them • Give secretaries as much information as you can so rumors don’t get started. Squash them when they do • Deal with troublemakers immediately • Help your team to be as well informed as possible so you aren’t giving out different answers to questions • Make sure that the link between assessments and training is obvious • Have a plan for your next steps so that assessment becomes a regular part of training and development

31. What Can Assessments Bring to Your Training Programs? • A model for further training efforts • Makes training an ongoing process rather than a one-time event • Gets your users to accept responsibility for their training • Gives you the potential for assessing other groups • Credibility- you now have numbers to impress people with • A chance to work on a common goal with several groups • The opportunity to define best practices and create consistency • A sense of accomplishment for users and trainers

32. Questions?

33. Training Results You Can Measure:Why Conducting Secretarial Skills Assessments Is Worth Your Effort March 13th, 2009 Corby GuentherSenior Technology Training and Development ManagerBingham McCutchen LLP corby.guenther@bingham.comBay Area Legal Trainers Organization http://groups.google.com/group/baltorg