1 / 43

Soft Skills Systems / Teaching / Assessment

Soft Skills Systems / Teaching / Assessment. Kostas Kechagias. Presentation in 2 nd MASS meeting Work Package 4, Needs Analysis / Piloting. June 2010. Contents. Skills and Skills Frameworks Teaching Skills and Competences Skills Assessment Experimental Methodology Conclusions/Homework.

kiora
Télécharger la présentation

Soft Skills Systems / Teaching / Assessment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Soft SkillsSystems / Teaching / Assessment • Kostas Kechagias Presentation in 2nd MASS meeting Work Package 4, Needs Analysis / Piloting June 2010

  2. Contents • Skills and Skills Frameworks • Teaching Skills and Competences • Skills Assessment • Experimental Methodology • Conclusions/Homework

  3. Skills and Skills Frameworks (1/13) • …towards definitions (DeSeCo) • great interest for identification of common skills • must be connected to success in life • the definition should address how they are used • Definition: skills or abilities individuals need in order to achieve success in life, within the context of their socio-cultural milieu, through adaptation to, shaping of, and selection of environments • Often they are used interchangeably or in a vague sense

  4. Skills and Skills Frameworks (2/13) • a proliferation of efforts to develop lists of skills, knowledge, and competencies necessary for success in the workplace and society • “basic skills” are not sufficient for success in the workplace • …“employability skills” • …“enabling skills,” “generic skills,” “core skills,” “key competencies,” “ essential skills,” and “necessary skills.” • methodology: start from a broad definition and then survey or observes workers, supervisors, and experts to determine what skills are common

  5. Skills and Skills Frameworks (3/13) • SCANS • 1991, US president and Education Secretary, America 2000 • seven skill groups • Influence: Organizational Effectiveness and leadership • Group effectiveness: Interpersonal skills, negotiation and teamwork • Personal management: Self-esteem, goal setting/motivation, and personal and career development • Adaptability: Creative thinking and problem solving • Communication: Listening and oral communication • Competence: Reading, writing and computation • The foundation: Learning to learn.

  6. Skills and Skills Frameworks (4/13) • Michigan • Center for Career and Technical Education, Michigan State University • human relations skills • Positive work attitude, • Teamwork • Listening • Honesty and Idengrity • Demonstrate initiative • Handling pressure & tension • Communication • … • conceptual skills • technical skills

  7. Skills and Skills Frameworks (5/13) • Scottish Qualifications Authority, 2008 • Problem Solving • Communication • Numeracy • Information and Communication Technology • Working with Others • Component: Working Co-operatively with Others • identifying, with support, your own role and the roles of other people, and helping to achieve a shared goal • to: • analyzing the roles and behavior of others and adapting your own behavior to deal with the complexity of changing and challenging dynamics

  8. Skills and Skills Frameworks (6/13) • DeSeCo (Definition and Selection of Competencies) • OECD, 1997 • A competency is more than just knowledge and skills. It involves the ability to meet complex demands, by drawing on and mobilising psychosocial resources (including skills and attitudes) in a particular context.

  9. Skills and Skills Frameworks (7/13) • Using Tools Interactively • use language, symbols and text interactively • use knowledge and information interactively • use technology interactively • Interacting in Heterogeneous Groups • relate well to others • cooperate • manage and resolve conflicts • Acting Autonomously • act within the big picture • form and conduct life plans and personal projects • assert rights, interests, limits and needs

  10. Skills and Skills Frameworks (8/13) • ALL (Adult Literacy and Life skills survey project) • OECD, 2005 • employability skills derived from job and task analysis and psychological theories of intelligence • scientifically convincing theory and an established literature • accompanying tradition of measurement • could be learnt and, therefore, taught • had a direct impact on the social, health, educational or economic life chances of individuals

  11. Skills and Skills Frameworks (9/13) • European Qualification Framework • “Education and Training 2010” (2004) • communication in the mother tongue • communication in foreign languages • mathematical competence and basic competences in science and technology • digital competence • learning to learn • social and civic competences • sense of initiative and entrepreneurship • cultural awareness and expression

  12. Skills and Skills Frameworks (10/13) • MASS Framework • we don’t have much information • don’t seem to fit in any studied framework

  13. Skills and Skills Frameworks (11/13) • No grouping of skills is provided • Some skills may be fit in more than one categories in other skill frameworks, e.g. • Motivation • Taking on new challenges, working hard to achieve goals, thinking of new ways to do things • Under SCANS contains elements from categories • Personal Qualities” (taking on new challenges) • “Thinking Skills” (thinking of new ways to do things) • Seems to be closely related to “Proffesionalism” • Working to a high standard, being consistent in attitude (not allowing emotions or personalities to influence you)

  14. Skills and Skills Frameworks (12/13) • Most frameworks are concerning higher education graduates • One may argue that the target group is different, so the skills to be considered are different • however, this should be established in the grounds of empirical research of literature • The skills should be defined in such a way as to minimize overlap • Failure to do so may have impact to teaching, but mainly to assessment.

  15. Skills and Skills Frameworks (13/13) • Questions: • What is the framework on which the MASS skills are based? • What justifies the selection of the particular skills as the most important ones? • If there is not a given one, referred from the literature, what are the empirical studies that support the selection? • Are the skills chosen in such a way as the interdependencies are minimized? • Is the selection justified from the view point of the assessment?

  16. Teaching Skills and Competences (1/9) • Gonczi, 2003 (DeSeCo) • The best examples: active independent learning, especially in ways that simulated contexts they might experience in later life • those that integrated the key competencies into the teaching of other material, rather than attempting to teach them as stand-alone entities • seemed to be more success in student outcomes when the key competencies were made explicit • in many instances, it was not possible to separate out the key competencies in the process of teaching and learning. • very few of the projects in vocational education specifically assessed the acquisition of key competencies directly. The most comprehensive attempt to assess them was via a portfolio of evidence which students collected for themselves.

  17. Teaching Skills and Competences (2/9) • Welsh & Canning, 2003 (SQA) • The prospect of the isolation of Core Skills, as a separately taught and assessed subject, would appear to be unpopular with candidates. From the literature, it can be seen that this is also unpopular with employers. • Completely separate delivery and assessment often not popular with candidates as it’s ‘tantamount to a return to school’

  18. Teaching Skills and Competences (3/9) • Shuman, et. all., 2005 (ABET) • The choice of team task depends on the educational objectives • if the objective is to have students experience a specific aspect of teaming such as brainstorming, specific group exercises new ideas can be used • if the objective is for students to experience a broad range of team processes and behaviors, then more complex activities are suitable

  19. Teaching Skills and Competences (4/9) • Shuman, et. all., 2005 (ABET) • Two guiding principles should be followed in choosing activities: fidelity and complexity • Fidelity: similarity of the training situation to the students’ present and future working conditions. • The higher the fidelity, the more superior the transfer of learning to the workplace • Complexity is defined by two subfactors: task interdependence and cognitive effort. The more complex the activity, the more team skills are required by the participant.

  20. Teaching Skills and Competences (5/9) • Shuman, et. all., 2005 (ABET) • advantages and disadvantages: • fidelity/high complexity activities: • most resemble real workplace” conditions, • typically are more difficult for the instructor to manage, • resource intensive, • time consuming for the student. • lower in fidelity/ low complexity activities • typically more structured and easier to administer, • may be perceived as less relevant by the student, • the experience having less of a learning impact. Finally, • may not challenge the team nor provide the environment necessary for intense interaction among team members

  21. Teaching Skills and Competences (6/9) • Teaching methods (Szul, 2002) • Coaching • Project Work (e.g. Newsletter) • Portfolio • Job-shadowing experiences • Videotapes/Observations • Interviews • Case studies • Role playing

  22. Teaching Skills and Competences (7/9) • MASS Teaching • seems to be standalone course • justification needed • Interdisciplinary • 18 weeks of approximately 4 hours/week • partners should be consulted for the feasibility

  23. Teaching Skills and Competences (8/9) • The activities that the 1st learning byte contains belong to the following categories: • Presentation: 4 times • Group discussion: 2 times • Directed Discussion (Class Discussion): 6 times • Quiz: 2 times • Observation: 1 time • Individual work: 1 time • Personal discussion with the teacher: 1 time • Mainly teacher centered designed • “Hopefully, your grades will be improved”

  24. Teaching Skills and Competences (9/9) • the design of the lessons seems a lot like the designs of the vocational training • Question about their efficiency • Excellent quality of materials, within the concept • instruction based not project based • question whether we train on the knowledge of soft skills rather than soft skills themselves • In some places, as in formation of student groups, more elaboration is needed

  25. Assessment (1/14) • indicators and outcomes • indicators are the means by which we can measure whether the outcomes have been achieved • achievements which may ‘indicate’ acquisition or progress towards an outcome • Not all indicators will be suitable for all target groups, and some will be target group specific.

  26. Assessment (2/14) • Methods for collection of information on soft outcomes • Individual action planning, personal action planning and goal setting • Reviews between trainers/assessors and clients to record soft outcomes • Daily diary or personal journal • In-depth reflection during or after the course • Recorded observations of group or individual activities • Presentation of material in a portfolio • Tests

  27. Assessment (3/14) • requirements from the tools • reliability • Validity • Requirements from the definitions • two ends • so general that it becomes totally abstract and refers more to a system of values than to abilities • less comprehensive in one’s ambitions and is content with assessing a few specific competencies in a necessarily limited context the

  28. Assessment (4/14) • Competencies do not exist in themselves; they can only be demonstrated when an individual is acting in a specific context. • pieces of research have shown that the subject’s performance is largely dependent on the contents of the task, on any previous training the subject may have had, on the type of strategy resorted to, and on the formal characteristics of the task • the same tasks undertaken in different situations—such as work situations or job interview situations—can yield very different results

  29. Assessment (5/14) • Teamwork (Baker, et al., 2005) (ALL Framework) • Focus towards identifying the specific competency requirements of team members • “competencies” a cluster of related knowledge, skills, and attitudes that affects a major part of one’s job • is correlated with performance on the job • can be measured against well-accepted standards • can be improved through training and development.

  30. Assessment (6/14) • three types of competencies that are central for effective teamwork Cannon-Bowers et al. (1995) • team knowledge competencies, • the principles and concepts that underlie a team’s effective task performance • team skill competencies, • learned capacity to interact with other team members at some minimal proficiency level • adaptability, situation awareness, performance monitoring/feedback, leadership, interpersonal relations, coordination, communication, and decision making • team attitude competencies • an internal state that influences a team member’s choices or decisions to act in a particular way • Positive attitudes toward teamwork and mutual trust among team members

  31. Assessment (7/14) • ALL questionnaire for teamwork • Evaluation of teamwork requires observation • Target group: International, adults • Examines • knowledge about team skills, • attitudes toward teamwork • historical experience with teamwork

  32. Assessment (8/14)

  33. Assessment (9/14) • Zhuang, et. all., & Roberts, 2008 (ETS) • Questionnaire • target at high school students (mean age 16.10) undertaking Ford PASS courses • consists of reliable and qualitatively distinct factors • involves a variety of methods • has demonstrable validity evidence

  34. Assessment (10/14) • Contains • Self-report ratings (57 items) • cooperation with others (15 items), • advocate and influence (12 items), • resolve • conflict/negotiate (17 items), • guiding others (13 items). • Situational judgment tests (SJTs) • eight scenarios describing various situations and required participants to evaluate the effectiveness of four reactions to each situation • Teacher ratings

  35. Assessment (11/14) • MASS assessment • one might form the impression of an underlying assumption that each skill, can be assessed separately • Is it possible? Interrelated skills? • Some skills, as “motivation” is difficult to assess outside a particular framework • Need for clarification of definitions? • Development of indicators? • Framework?

  36. Assessment (12/14) • the main assessment tool proposed is as self reference questionnaire • students mark their “grades” against each skill, every week • They can see the grades they put to themselves in the previous weeks • “aim being that your scores will have improved” - biased toward to a false improvement of grades? • What about the students that they do not (or they thing they do not) improve? – motivation for more or drop out?

  37. Assessment (13/14) • three entities involved in the course, the students, the teacher(s) and the materials • Student self evaluation • Student evaluation from the students (peer evaluation) • Student evaluation from the teacher(s) • Teacher self evaluation • Teacher evaluation from the students • Material and course quality evaluation from the students • Material and course quality evaluation from the teachers.

  38. Assessment (14/14) • questionnaires are convenient, but others, like portfolio or in depth interviews may be considered • the partners should decide about the aims and the methodology of the evaluation • If we decide to confine ourselves in a given framework, the uses of readymade and tested tools are also possible • other aspects of influence • self-esteem and self-image

  39. Experimental Methodology (1/1) • The proposed methodology is the classical experimental one • Two groups, the experimental and the control one • Initial Assessment • Intervention • Final Assessment • What about long term results?

  40. Conclusions/Homework (1/3) • Does MASS framework needs clarification and justification? • Is the proposed schedule 18 weeks X 4 hours suitable for the partners? • Each partner should report on this • Suggest modifications • Two alternatives • All the same schedule – Comparable results • Different schedules – Test more than one variations of program to see which works best.

  41. Conclusions/Homework (2/3) • Teaching organization • How many students • How many classes • What age • What educational background • Suggested period of teaching • …what else? • Decision about assessment • Target of assessment • Ages of students • Assessment procedure: questionnaires only or something more? • Each partner studies and comments proposed questionnaires and maybe contributes with more • …what else?

  42. Conclusions/Homework (3/3) • Deadline? • What about end of June?

  43. Thanks for your patience!!!

More Related