1 / 21

Outline(1)

Use of Knowledge, Skill, and ability statements in Development Licensure and Certification Examinations. Wang, N., Schnipke, D., and Witt, E. A. (2005), Educational Measurement, Issues and Practice, 24(1), 15-33. Outline(1). Abstract Introduction

lani
Télécharger la présentation

Outline(1)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Use of Knowledge, Skill, and ability statements in Development Licensure and Certification Examinations Wang, N., Schnipke, D., and Witt, E. A. (2005), Educational Measurement, Issues and Practice, 24(1), 15-33

  2. Outline(1) • Abstract • Introduction • Job Analysis for the Development of Credentialing Examinations • A Framework Describing the Important Role of KSAs

  3. Outline(2) • Example 1: A National Nurse Aide Certification Examination • Example 2: A National Licensing Program for Real Estate Professionals • Summary

  4. Abstract • Addresses the important role of KSAs in developing and validating licensure and certification examinations, includes the use of KSAs • in linking job task survey results to the test specifications, • Transferring job task weights to test specifications, • Applying the results to the development of the test items. • Two examples are taken from nursing certification and real estate licensure programs.

  5. Keywords • Job analysis • KSAs • Licensure and certification exams • Test development • Subject matter experts (SMEs)

  6. Keywords(1) • Job analysis • 工作分析(Job Analysis)又可稱為職位分析，是指採用較科學而有系統的方法收集有關工作所包含的工作項目、任務、責任的各項資料，並據以分析擔任此一工作所須具備的知識(knowledge)、技術(skill)與能力(ability)。

  7. Keywords(2) • Subject matter experts (SMEs) • 主題專家 • 專擅於特定主題的個人；在知識管理的環境裡，主題專家被視為是權威，並且是該特殊主題相關知識領域的守門人。

  8. Introduction(1) • Licensure and certification examinations are developed to determine whether candidates have adequate levels of knowledge and skills to perform competently in their professions. • The use of job analyses as the primary basis of validity evidence for credentialing examination scores has been advocated by some organizations (ex. SEPT,EEOC)

  9. Introduction(2) • The most common approach of a job analysis to use a task inventory questionnaire. • Ask SMEs • Develop a survey questionnaire • Select a representative sample of practitioners • Have the survey respondents rate each task • Analyze the survey data and determine the relative importance of these tasks • Use the resulting quantitative measure of task importance to develop test specifications

  10. Introduction(3) • The task inventory approach is the most common method for job analysis for credentialing exams primarily because it can provide direct access to job-related information. • It is often the knowledge, skills, and abilities required for performing the tasks, rather than the job tasks themselves, that are tested by licensure and certification exams.

  11. Introduction(4) • Failure to demonstrate a linkage between job tasks and test content can result in test score that are not truly valid measures of job-related competency. • The current study advocates the establishment of such a linkage by building test specifications from KSAS that are directly connected to the job activities surveys by the job analysis.

  12. Job Analysis for the Development of Credentialing Examinations • Standards for Educational and Psychological Testing • This study suggestions the following four functions for a job analysis： • The method of job analysis should be derived directly from the designated test purpose. • The selected job analysis approach should be able to clearly define the content domain of the profession. • A linkage should be established between the job content domain and the contest domain of any tests that are used for licensure and/or certification. • The procedures used in the job analysis should provide evidence that exam scores derived from test items are valid indicators of professional competency.

  13. A Framework Describing the Important Role of KSAs • Define test content outline by conducting a systematic task-KSA linkage • Determine content outline weights by transferring task weights to KSA weights • Develop Test Items from KSA-based test specifications

  14. Roles of KSAs in Test Development

  15. Task-KSA Linkages

  16. Example of Computing KSA Weights Through Equal Distribution of Task Weights

  17. Example 1: a National Nurse Adie Certification Examination

  18. Example 2: A national Licensing Program for Real Estate Professionals

  19. Summary • This study advocates the use of KASs in establishing the linkages between job analysis results and test items when it is not practical to test items when it is not practical to test job tasks directly. • The important role of KSAs in developing and validating licensure and certification examinations is justified and demonstrated along with examples.

  20. Commentary: A Closer Look at Task analysis : Reactions to Wang, Schnipke, and Witt LaDuca, T. (2006), Educational Measurement, Issues and Practice, 25(2), 31-33

  21. Abstract • This article offers a critique in the form of our counterarguments： • The job analysis method must be appropriate to the nature of the work; • The capacity of the task analysis inventory method to describe a complex profession is questionable; • Task importance ratings are inherently deficient; and • Careful generation of KSAs do not assure systematic item development, which is also necessary in the chain of validity evidence.

More Related