Creating a Standard Arts Assessment: Written Evaluation vs. Project Based An Informal Case Study Ruth Currey ARE 6905: Research Trends in Art Education
Purpose Determine and evaluate the possibility of creating a National Assessment for computer-based visual arts classrooms Goals and Processes • Conduct an informal study that evaluates students on technological processes in both a • written format as well as a skills test format • Evaluate and consider the data collected from each form of assessment • Make preliminary conclusions concerning modes of assessment • Create a potential options for Technology-Based Art Classrooms based on Industry • Standard Software Personal Interest & Current Status • With Animation I classrooms packed to the brim, written exams may be the only • feasible option due to time constraints • Although National Exams such as the NAEP are available for a portion of the arts, these • exams do not cover any technology based skills or processes • Relevant competency exams are needed for technology based art classrooms • No data exists currently that accurately demonstrated technology based learning
Questions to Consider • Will creating a written standard assessment truly demonstrate the artistic knowledge of the students? • Can multiple choice tests identify a student’s artistic potential? • Students are currently allowed to enroll in an AP Drawing class based on PSAT, FCAT, and • Stanine scores in Reading and Math • Can a project based assessment be created? • What would it look like? • Would separate standards have to be created by artistic content area? • Would standards be set based on grade level? • Does the assessment tool measure what you need it to measure? • Realistically, can a set of National Standards be designed based on the art • making process? Possible Limitations to Consider • Assumption – Students will have the same sequence of classes to ensure receipt of all necessary • information • Depending on the school and guidance department, students are given the opportunity to skip • classes in a sequence due to scheduling • Assumption – Students will have access to the necessary supplies needed to meet the National • Standards for project based assessment art supplies and media availability change from school to school • Funding is not available to all schools in order to provide the exact same material at every • single school
Preliminary Case Study Examples Student A Computer Animation I – Written Skill Exams Process Sequential Order Quiz Score: 12/12 (100%) Fill in the Blank Process Quiz Score: 11/12 (92%)
Preliminary Case Study Examples Student B Computer Animation I – Written Skill Exams Process Sequential Order Quiz Score: 3/12 (25%) Fill in the Blank Process Quiz Score: 2/12 (17%)
Project Based Assessment Animation I Student A Ball Bounce using Motion Guides Score: 49/50 (98%) Oral Skills Demonstration Quiz Score: 23/25 (92%) Animation I Student B Ball Bounce using Motion Guides Score: 47/50 (94%) Oral Skills Demonstration Quiz Score: 25/25 (100%)
The Teacher as Stakeholder in Student Art Assessment and Art Program Evaluation By: Charles Dorn Art Education, 55(4), pp. 40-45 “What cannot be tested cannot be taught, the arts in the near future may face being left out of the curriculum.” (Dorn, 2002, p. 40) • Article Discusses • National Education Goals and National Art Standards • NAEP Assessment • Authentic Assessment • What teachers teach and assessment based on information taught • What needs to be taught • Models for assessing arts performances • Recommendations • Teachers develop their own authentic assessment instruments • Teachers develop school and district assessment plans using a peer review process • Teachers develop ways to document student progress and establish sensible and appropriate • record keeping systems that will meet the agreed upon goals of the district and state “In the end, we must realize that US school children are not equal in their aesthetic abilities and US schools are also not equal in the aesthetic opportunities they provide. But if we can at least entertain the possibility that either one or both of these conditions are reversible, our best hope lies in deciding what it is that kids need to know and be able to do and make that the primary focus in reforming schools and schooling.” (Dorn, 2002, pp. 44-45)
Performance Assessment in the Arts By: Robin E. Clark Kappa Delta Pi Record, Fall 2002 “Despite the difficulty of assessing a non quantifiable field like art, teachers in the arts and other areas should creatively embrace different methods of performance assessment to evaluate student work more accurately.” (Clark, 2002, p. 32) • What is Performance Assessment? • In a standards-based curriculum, student performance generally is expected to be the most important • indicator that learning is taking place • Teachers are required to provide evidence of learning based on multiple assessments • According to Clark, art should be leading the way in performance based assessment • Process • The entire range of activities required to produce a work of visual or performing art from • concept to creation • Product • The aesthetic quality of the student’s finished result of work of art • Art teachers typically recognize the need for both forms of assessment • A finished product or performance does not necessarily reveal skills, knowledge, creative • thought, and personal meaning of the process that created it “A balanced approach to grading considers both student engagement in the process of creation and the quality of the resulting product or performance” (Clark 2002, p. 30).
Performance Assessment in the Arts By: Robin E. Clark Kappa Delta Pi Record, Fall 2002 • Assessment Methods • Portfolios: self-assessment and collections of student work over a period of time • Critique: causes students to reflect on his or her work and hypothesize a means for improvement • Demonstrations: makes student knowledge level visible based on teaching their peers • Working Exhibitions: opportunities to actively discuss their all works on display • Annotated Sketchbooks: documents essential points along their creative process • Other forms of assessments can be modified to accommodate performance based • assessment • Written research papers, presentations, class discussions, tests quizzes and exams • Time-Efficient Assessment • Teachers are open to suggestions and tips regarding time management particularly with assessment • Well-constructed rubrics • Provide a way for teachers to evaluate performance easily without compromising divergent • qualities of the individual creative processes as well as the final product performances • Needs to be a balance between subjective and objective elements • Essential to have a clear statement regarding what students should know by the end of the • project • Students should receive copies of the rubric prior to the start of the lesson “Students’ performance can offer solid proof that they know and are able to do what you are trying to teach them and what the standards require” (Clark, 2002, p. 32).
Measuring Student Learning in Art Education By: Donald D. Gruber Art Education, September 2008 “No single aspect of assessment can provide a representative and accurate measure of student learning in art” (Gruber, 2008, p. 42). • Difference between Evaluation and Assessment • Evaluation • Encompasses the global aspects of the curriculum and measures program efficacy • Assessment • More tightly focused measurements at the level of the individual student and their interactions • within the art program • Developing Assessment Criteria • Process Criteria • Considering the development of learning and growth within a program • Teachers report attendance, effort, class participation, homework, etc. • Reliable measures of student growth and establish a baseline to gauge growth • Product Criteria • Significance of where students are is more important than how far they have come • Overall assessments, final exams scores, and final products • Provide a valid estimate of student learning as an indication of current ability and knowledge
Measuring Student Learning in Art Education By: Donald D. Gruber Art Education, September 2008 • Methods of Assessment • Written Tests • Most widely used source of assessment information • Researchers warn about relying on test validity due to their inability to measure aesthetic • response • Observations • Consistent surveillance during all phases of the lesson including • Performance • Attitude (level of engagement / sharing discoveries / attention to task) • Work habits and behavior • Other observational strategies • Checklists: lists of attributes of projects that can be compared • Rubrics: detailed guides for scoring student products, performance, and portfolios • Portfolios • Formative Portfolios: contain all aspects of a particular lesson from initial conceptualizations • through research and revisions to final products • Summative Portfolios: contain collections of finished products that give indications of abilities at a • given point in the instruction (usually at the end) • Assessment Strategies • Must consider objectives • Unproductive to assess without regard to stated objectives • Measure how well the objectives of each lesson are being met • Based assessment on a set of established standards
Data Charts • Students we separated based • on educational status • Gifted=G • Mainstream=M • ESE=E • Numbers reflect points earned • for each assessment method • A total of 95 students are included • in this study with the breakdown • being: • Gifted = 10 students (10%) • Mainstream = 70 students (74%) • ESE = 15 students (16%) *Final analysis will include charts and graphs for each subdivision of students
Gifted Breakdown Sequential Order Average Score: 12/12 (100%) Fill in the Blank Average Score: 11/12 (91%) Overall Written Assessment Average: 96% Ball Project in Flash Average Score: 48/50 (96%) Oral Skills Test Average Score: 24/25 (96%) Overall Project Assessment Average: 96% 0% Differential • No student scored below an 84% on any • written assessment • All scores for project based assessment • ranged between 92% to 100% • Gifted students demonstrated a level • consistency across assessment modes
ESE Breakdown Sequential Order Average Score: 5/12 (41%) Fill in the Blank Average Score: 4/12 (33%) Overall Written Assessment Average: 38% Ball Project in Flash Average Score: 40/50 (80%) Oral Skills Test Average Score: 21/25 (83%) Overall Project Assessment Average: 82% 44% Differential • Scores ranged from 8% to 66% on written • assessments • Student scores increase significantly for • project based assessment ranging from 72% • to 100% • Students successfully demonstrated learning • gains primarily in animation processes through • project based assessment
Mainstream Breakdown Sequential Order Average Score: 8/12 (66%) Fill in the Blank Average Score: 8.6/12 (72%) Overall Written Assessment Average: 69% Ball Project in Flash Average Score: 41.5/50 (83%) Oral Skills Test Average Score: 22.25/25 (89%) Overall Project Assessment Average: 85% 16% Differential • Scores for written assessment modes • exhibited fairly large learning gaps between • the two poles ranging from 16% to 100% • Project based scores had a significantly • lower learning gap ranging from 75% to 100% • Scores demonstrated a statistically significant • increase from written assessment to project • based assessment modes • Data exhibits a consistent incline as the mode • of assessment shifts from written to project
Preliminary conclusions • Preliminary research indicated that the most effective means of assessment was to • maintain a balanced approach • Based on the data collected, I would be inclined to agree • Gifted students seemed to excel across the board (96% for both written & project based) • Mainstream students maintained a steady incline as the assessments moved towards • project based (69% written / 85% project based) • ESE students responded significantly higher when assessed through project based (38% • written / 82% project based) • Data also indicated multiple occurrences where ESE students out performed mainstream • students on project based tasks • Written assessment seems to yield lower scores for mainstream and ESE students • Preliminary data indicates failing scores for written assessments, however the point • values are lower in comparison to the project based values • For this reason, the data was informally compared to the final grade received • When comparing results to the final grades, most students from each subgroup • were able to successfully complete the course with a “C” or higher • However… • Does adding the written assessments automatically decrease their final • grade potential? • Should that matter? • Consideration also needs to be give to other potential factors such as lack of reading or • writing skills
Possible Solutions • Written Assessment • Lends itself well to overcrowded introductory level courses • Approximately two-thirds of the subgroups struggled when faced with written assessments • Project Based Assessment • Does not lend itself to individualized work due to overcrowding and lack of available technology • Success rates are higher for all subgroups • Based on the data collected, the best option would be to create a balanced curriculum that requires • multiple modes of assessment in order to provide the most opportunities for success for the students • A Balanced Combination • Students will be assessed in a manner that reflects the principles of differentiated instruction • Students will be exposed to written as well as project based assignments in order to fully • demonstrate where their strengths and weaknesses lie • Students will have the opportunity to work with partners to compensate for overcrowding, • However, individual oral exams as well as project accountability must be built into the rubric • and the curriculum • Possible Limitations • Provides the best opportunity for students to succeed and demonstrate their learning • gains and individual skills if the value attached to the assessment is balanced over the entire • curriculum • Does not lend itself to overcrowded classroom due to the time needed to log in each individual • student for individual assessment. • Exam periods prove difficult
Personal Learning Gains & Reflection • Statistics and I have decided to maintain our love hate relationship • Teaching and assessing students effectively are trial and error processes • When working in a relatively new aspect of a content area there are no absolutes • More research needs to be conducted in the field of technology based art • education before any definitive recommendations can be made • Just from my own personal research, I have learned such a great deal about how • my students function in my classroom and how they respond to the assessments • and activities I present • If more teachers took the time to consider how their assessments are actually • evaluating their students, more effective assessments may be possible • Even with all of the data that was collected, I am still undecided on how to proceed • with the assessments for the Animation I course • The written assessments demonstrate a task oriented understanding based on • the representation and interpretation of language • The project assessments demonstrate a literal grasp of the information and exhibit • direct learning gains based on a tangible product
References Clark, R. (2002). Performance Assessment in the Arts. Kappa Delta Pi Record, 39(1), pp. 29-32. Dorn, C. (2003). Models for Assessing Art Performance (MAAP): A K-12 Project. Studies in Art Education, 44(4), pp. 350-370. Dorn, C. (2002). The Teacher as Stakeholder in Student Art Assessment and Art Program Evaluation. Art Education, 55(4), pp. 40-45. Gruber, D. (2008). Measuring Student Learning in Art Education. Art Education, 61(5), pp. 40-45.