1 / 21

Assessment Plan and Notes on the ABET Program evaluator training

Assessment Plan and Notes on the ABET Program evaluator training. J. Fernando Vega-Riveros, Ph.D. Dept. of Electrical and Computer Engineering University of Puerto Rico . Objective of presentation. Present issues and comparative analysis of assessment approaches and ABET criteria

Sophia
Télécharger la présentation

Assessment Plan and Notes on the ABET Program evaluator training

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessment Plan and Notes on the ABET Program evaluator training J. Fernando Vega-Riveros, Ph.D.Dept. of Electrical and Computer EngineeringUniversity of Puerto Rico ECE Department Meeting

  2. Objective of presentation • Present issues and comparative analysis of assessment approaches and ABET criteria • Promote discussion of issues as they pertain to Department INEL/ICOM • Make a decision about department assessment instruments and methods ECE Department Meeting

  3. Rationale • A unified assessment approach for our two programs would would reduce administrative and academic burden on faculty, students and staff. • Same instruments and tools used for the whole department ECE Department Meeting

  4. Accreditation timeline INEL and ICOM accredited till 2009 Request re-accreditation to ABET Submit Self-Study Report ABET Visit Design metrics, thresholds and sampling method Define assessment instruments Conduct assessment and close the loop Assessment Plan December 2005 Summer 2006 Summer 2007 Today July 2008 January 2008 December 2009 ECE Department Meeting

  5. Assessment: act of collecting data or evidence that can be used to answer classroom, curricular. Or research questions. Assessment method: Procedures used to support the collection of data for assessment purposes. Evaluation: refers to the interpretations that are made of the evidence collected about a given question. Assessment and evaluation1 1 Olds, B.M.; Moskal, B.M. and Miller, R. Assessment in engineering education: evolution, approaches and future collaborations. ASEEJournal of Engineering Education. Vol. 94 No. 1. Jan. 2005. pp 13-25 ECE Department Meeting

  6. Types (components?) of assessment Assessment of outcome achievement in a given academic term (snapshots) Formative Assessment Transversal Assessment Summative Assessment Longitudinal Design Tracks outcome achievement over time ECE Department Meeting

  7. Goal of assessment • Measure outcome achievement (a-k) and how these outcomes are developing along and across the curriculum • Validate interventions, e.g. curricular revisions, improved or novel pedagogies, new courses, course restructuring, etc. • Collect data for quality assurance and continuous quality improvement • Realistic and minimize burden ECE Department Meeting

  8. Program Educational Objectives Program will allow the graduates of the program to: • Obtain a broad educational experience necessary to understand the impact of electrical/computer engineering problems and solutions within a global and societal context; • Possess a combination of knowledge and analytical, computational, and experimental skills necessary to solve practical electrical/computer engineering problems; • Have adequate communication skills both as an individual and as part of a team; • Value the importance of lifelong learning • Be aware of contemporary issues and thus be able to make decisions taking into consideration professional and societal needs and ethical implications ECE Department Meeting

  9. Program educational outcomes Graduates of the program will have the: • Ability to apply knowledge of mathematics, science and engineering necessary to carry out analysis and design appropriate to electrical/computer engineering problems; • Ability to design and conduct experiments as well as analyze and interpret data; • Ability to design computer systems to meet desired needs; • Ability to function in multidisciplinary teams; • Ability to identify, formulate, and solve engineering problems; ECE Department Meeting

  10. Program educational outcomes (cont) • Understanding of professional and ethical responsibility • Ability to communicate effectively; • Broad education necessary to understand the impact of engineering solutions in a global/societal context; • Recognition of the need for and ability to engage in lifelong learning • Knowledge of contemporary issues • Ability to use the techniques, skills, and modern engineering tools necessary for engineering practice. ECE Department Meeting

  11. Current Assessment Approach (ICOM) • Instruments • GPA (Transcript Analysis) • Student, alumni and employer surveys • Metrics and satisfaction thresholds • Items related to grades are considered satisfactory it they are above 3.0 (B grade) • Items related to satisfaction rating are satisfactory if • Above 3.0 (weight of a “satisfied” response in surveys); or • Above average satisfaction of all items in same category ECE Department Meeting

  12. Issues of current approach • Minimum burden on faculty • High burden on accreditation team (coordinator) • Uses surveys, practice that ABET discourages for outcome assessment. Surveys can be used as supportive information but depends on perception not on actual outcome achievement3 • Some thresholds move depending on population and may not be set according to more explicit program criteria. 3 ABET 2005 Program Evaluator Training Materials (EAC Training, Approved December 2004). ECE Department Meeting

  13. Issues of current approach (Continued) • GPA does not discriminate among outcomes of a given course; it gives some indication of achievement of the set of outcomes set for a given course or courses. • ABET has some concerns about use of course grades4. 4 Guidelines to Institutions, Team Chairs and Program Evaluators on Interpreting and Meeting the Standards Set Forth in Criterion 3 of the Engineering Accreditation Criteria. http://www.abet.org/Linked Documents-UPDATE/Program Docs/EAC Guidelines or Criterion3.pdf. ECE Department Meeting

  14. Assessment methods2 • Descriptive designs • Surveys • Interviews and focus groups • Observation • Conversational analysis • Ethnographic studies • Experimental designs • Randomized controlled trials • Matching • Baseline data • Longitudinal design • Post-test-only design 2Olds et-al Op. cit. ECE Department Meeting

  15. Approaches to outcome assessment ECE Department Meeting

  16. Approaches to outcome assessment (Continued) ECE Department Meeting

  17. Approaches to outcome assessment (Continued) ECE Department Meeting

  18. Assessment Proposal • Establish cycles of assessment for courses • Sampling of course material (no need to be exhaustive, just representative and statistically significant) • Mapping of course material and educational outcomes (may need collaboration of course instructor) • Measurement of outcome achievement using course material and outcome mapping. • No course is assessed every semester (unless something exceptional is found). Round-Robin strategy. • Not all sections of a course need to be assessed. Assessment plan will establish cycle of sections assessment to minimize burden on faculty members and distribute assessment load as evenly as possible. • Will use professional licensure examination as a component of summative assessment and benchmarking • Results reviewed by area committees and final results send to steering committees ECE Department Meeting

  19. Assessment Proposal – developing the assessment plan • Determine who we will assess: • Educational objectives: graduates between 1 and 3 years. • Educational outcomes: current undergraduate students in the Department. • Establish a Schedule for Assessment. Proposed re-accreditation timeline. • Determine who will interpret results: Area committees. • Interpret how results will inform teaching/learning and decision making: Steering committees. • Determine how and with whom we will share interpretations: Department, College, Office of Continuous Improvement and Assessment, ABET, Middle States, CES through periodic reports and self studies. • Decide how our department and institution will follow-up on implemented changes: periodic assessment and continuous program quality improvement. ECE Department Meeting

  20. Assessment Schedule Publish Assessment Schedule 1st Assessment Sampling 2nd Asmt Smpl and start closing loop 3rdAsmt Smpl and continue closing loop 4thAsmt Smpl & finish asmt. cycle Committee Evaluations Committee Evaluations Committee Evaluations Submit Self-Study Report Request re-accreditation to ABET ABET Visit July 2008 December 2009 January 2008 January 2006 June 2006 December 2006 June 2007 December 2007 ECE Department Meeting

  21. Questions, discussion and decision ECE Department Meeting

More Related