Dedra Demaree Oregon State University

1. Bridging Gaps: integrating research expertise with curricular development aimed to synchronize upper division course goals with our large introductory classes Dedra Demaree Oregon State University

2. Background: • Physics PhD research emphasis in Physics Education Research (PER): • Focused on writing to learn issues • Thesis: TOWARD UNDERSTANDING WRITING TO LEARN IN PHYSICS: INVESTIGATING STUDENT WRITING • Hired to lead introductory course reform at OSU • Current intro courses are ~250 students per section • Under-staffed, can not easily reduce class sizes • Algebra-based, calc-based, and non-science classes • My primary focus is implementing and assessing course changes

3. Do We Know if Writing is Helpful in Physics Courses? • No one questions the benefits of educating people to write • but why take the time to do it in the curriculum? • and why explicitly in physics? • There is no clear evidence in the literature to show the effectiveness of writing to learn! • Most writing studies are entirely qualitative and not controlled

4. Why MIGHT Writing be Helpful? • everyone raised their hands when polled if they think writing helps learning • (at American Association of Physics Teachers conference) • Students interviewed state “writing helps learning” • Students do better with a positive epistemology • Writing involves logical argumentation structure • Writing helps structure conceptual understanding? • Maybe writing = active engagement?

5. Why Might Students NOT Learn? • Struggling with content and writing is overwhelming • Issues with activating and managing their knowledge resources (research exists to support this) • Ideas can’t be organized if they aren’t already present in some form • Are writing activities striking the right balance? • Students may not be reflective when writing • Can we generate writing assignments that force reflection?

6. What Does Writing “Research” Tell Us? • How writing could help learning: • Ideas are transformed while writing • Rhetorical goals arerefined while writing • Literature provides minimal evidence to support these • ‘Knowledge-telling”: novices tell what they know - experts plan, write, and revise • Novices -> cosmetic changes • experts -> goal-oriented revisions • How do our students approach writing and revision? • How can we quantitatively study this? • Develop methods for tracking and coding writing to allow for controlled studies of the effects of writing in the curriculum

7. Initial studies and results: • Collaborated with English Department to do controlled test of effect of writingand writing instruction on physics content knowledge • Students who wrote did better on post-lab quizzes compared to students who did traditional activities • But no difference on lecture quizzes and exams • Writing instruction impacted physics quality in essays • No difference was measured outside their writing • Difficult and time consuming to quantify writing quality – need to find better ways to study this!!

8. Tracking Writing in Physics by Inquiry: • Force revisions within the assignments: • Weekly essay with homework • 1st draft 250 words – 2nd draft < 125 words! • Trying to force “major restructuring in your head, deciding what’s important and what’s not” (Scott Franklin, RIT) • Create tracking program for capturing details: • Obtain text file of their essay and a separate log file tracking writing events • Track pauses, additions, deletes, locations, & times • Developed by Dr. Lei Bao and members of the Ohio State U. Physics Education Research Group

9. Entering the Text (What Students See): Notice the program tracks and displays a running word count. It also saves the student’s name, email address, their section, and which assignment is being submitted

10. The Log File: • Saves a snapshot of the text each time a student pauses, backspaces, deletes, or moves the cursor • Indexes each event, gives the time, what type of activity the student is doing, the text snapshot, and the cursor location • Tags include: Typing, Backsp, Naviga, Delete, Pa{s} (pause and length in seconds), <CU> (cursor location) • Example: 13 10:57:027 AM: Typing A circuit is all <CU> 14 10:57:033 AM: Backsp A circuit <CU> 15 10:57:036 AM: Typing A circuit is the flow <CU> 16 10:58:043 AM: Pa{67} 17 10:58:053 AM: Backsp <CU>

11. Quantitative Information we can get From the Data: • Do students mostly write new content, or do they go back and revise while or after they write? • When students revise do they cut in bulk and rewrite, or do they modify existing text? • How often students write vs. edit or pause • How much do students work after the word limit? • Do observed behaviors match self-reports from interviews? • Can look at a lot of writing at once, but • Can’t automate information on the quality of the revisions!!

12. Student with low final exam but good essay content – 2nd draft: first rephrased needed content needed then cut extraneous text. Then did detailed editing pass through the entire essay, then one last check Has clumps of edits around specific text (she reported struggling with some ideas) Relatively sophisticated revisions!

13. Results and conclusions: • See evidence of novice vs. expert behavior… High Exam Scoring students had • more revision events, higher essay grades, more edits in the middle of their essays, a higher percent of phrase-level edits (between 2-6 words in length) • We see no evidence that writing behavior changes with practice • We find no clear predictors based on tracked behavior for which essays will be good • Developed a valuable new tool and begun to characterize student behaviors • Need to apply this test to controlled writing studies

14. Course reform ideas: • Students out of traditional introductory physics: • Minimal conceptual understanding • “Plug-and-chug” problem solving skills • Worse attitudes than when they registered • Interactive-engagement is more effective • Sophisticated epistemologies are encouraged when students are metacognitive • We gain more with focus on Higher-order learning goals • Traditional lecture halls do NOT encourage students to build their knowledge! • How to improve this in a large-lecture classroom?? • Need an interactive environment!

15. Our plan for Calc-based sequence: • Currently have: • 3 h lecture, 3 h lab, 1 h optional recitation • 250 people per lecture, 30 per lab with 1 TA • Change to: • 2 h lecture, 4 h activity-based learning in 2 h blocks, possibly keep 1 h optional extra help time • Possibly have lecture on M and F – keep all students at same pace in activity sections • 210 people per lecture, 70 per activity section with 1 senior instructor/TA and 1-2 TAs/undergrads

16. Why lecture at all? (besides staffing) Activities won’t be effective if students aren’t ready Prepare students for activity-based hours • Introduce definitions, Motivate students • Both can be done with readings • Create common language use, Show examples • Both can be done in activity-based hours Wrap-up after activity-based hours • Summarize important points, Look at capstone issues, Go over things people struggled with • All can be done in activity-based hours

17. Case Study: Iowa State University • Two lecture halls: one with fixed seats and one with swivel chairs – both nice and new • Use “Peer Instruction” • Swivel chairs made a measurable difference in learning gains: • Group discussions were physically easier • swivel lecture hall had higher percentage of correct responses after talking to neighbors • swivel lecture hall did 6% points higher on the final exam

18. Proposal for Weniger 151: • Swivel chairs for ease of discussion • Chairs and aisles organized to promote group work • Aisles for instructor access to all groups • Clump chairs in sections to minimize the number of needed aisles and maximize the number of seats • Boards (ideally smartboards) along the edges for groups to present ideas to the entire class • Multiple projectors up front so people can see from every angle • Camera to project demonstrations onto an overhead so everyone can see details

19. Weniger 151 layout details: • Students face forward in staggered chairs for lecture • Students can rotate to work in groups of 3-4 people • Each section has 4 rows – can form 2 rows of groups with people paired back to back • Instructor has access to each group • Minimally reduces the number of seats from 266 to just over 200

20. Model for new physics classroom: • Design a modern activity-based classroom • Design to fit our course goals/activities • Use modern technology to increase options • Test and assess new curricular ideas in this space • Inspired by SCALE-UP and echoing goals and activities in Paradigms • Start here because we know this works

21. Can we do better than this? • Lessons from PKAL (Project Kaleidoscope): • SHOW VIDEO (made by a KSU Anthropology class) • Students… • Want to build community • Use informal learning spaces • Work more on online activities • Rely on multi-tasking • Education community… • Thinks about green concerns • Highly values activity-based learning • Knows the importance of assessment • Emphasizes the use of technology

22. Building Flexibility for the future • A space that invites different types of activities • Floor that allows for ease of making new configurations • Technology that promotes collaborative work • Possibility of a design/work area • Cabinets that can be easily moved later • Flexible lighting, power, and media • Whiteboards on wheels • A window into and out of the room

23. Paradigms program • ~10 years of reforming upper division physics • Award-winning with Consistent NSF funding • Corinne Manogue just won the AAPT undergraduate teaching award • Team-based reform efforts unanimously approved by whole department • Brings active-engagement into advanced courses • Integrated lecture/lab/discussions • Group work • Extensive use of small and large whiteboards

24. Our Approach • Build on Paradigms expertise and borrow and adapt materials developed by other schools • Find goals that fit the needs of the students in their majors • Problem solving, group work… (ABET) • Find goals that fit the needs of physics majors as they segue to upper division • Earlier activity-based experience, more sophisticated problem solving, fit current need of more data analysis skills • Build our goals into the materials

25. Goals for New Curriculum: • Model “real” scientific behavior • Develop scientific skills - Have students: • Reflect on how they know what they know • Actively reconcile their knowledge • Understand the applicability of their models • Integrate simulations with experiments to explicitly address models and simplifications • Have students design and analyze their own experiments – teach them to build knowledge • Teach data analysis in the labs – build this in to fit current lack in overall program • Understand estimations and approximations • Integrate goals into exams and homework assignments

26. Testing some ideas in energy course • Want students prepared for lecture • Integrate pre-class reading assignments and quizzes (following JITT model) • Use existing technology – blackboard is powerful • Want to develop discourse skills – apply concepts to have “real” debate about issues • Use class time to scaffold up to sophisticated discussions • Use online tools for collaborative writing – “Wiki” • Group info gathering and posting then online discussions

27. Ph212 homework Problem Solving Guide • Understand and restate the problem • Read. Read the problem carefully. What are the key words? What information is given? What might you need to know in order to solve this? Explicitly state (in your own words) what is the problem asking including clarifying the problem statement. For example, if the problem states when will the two cars collide, you can state when will the two cars have the same coordinates for x and t. • Visualize. Visualize the situation described with a mental picture. What are the important features of the situation? What physically might happen? Think about what physics might be involved? (Repeat steps 1.a and 1.b as needed until you’re ready for step 1.c  ) • Simplify. Think of what assumptions you can make: can you ignore the size of the objects and consider them particles? Can you ignore friction? (Usually if the information about some properties of objects or interactions is missing from a problem statement, this means it is not important and you can ignore it.) In your homework you must explicitly state how this simplifies the problem – for example if you are ignoring friction in a collision it means you will be using momentum conservation for the system. • Picture and translate. Translate the text of the problem into a picture – record all given quantities in the picture and identify symbolically (name!) the relevant variables and unknowns. Choose and show the coordinate axis(es). Explain your picture with words if that makes it more clear. (Sometimes this step can be skipped and you can combine it with step 2.b – but only if you are very confident with the other steps.) • Devise and explain the plan • Determine what concepts/laws apply. Think what physics concepts are involved and which will be more helpful to solve the problem. For example, think whether the problem involves concepts of energy or force. Explain why you made the choice of this (these) particular physics concept(s). You may want to refer to 1 c. here, as in the example given there. • Represent physically. Represent the situation with the appropriate type of physical representation. This can be a free-body diagram, an energy bar chart, a ray diagram…. (If you skipped step 1.d, you must record all the given quantities and symbols for relevant variables and unknowns here.) • Represent mathematically. Use the physical representation to construct a mathematical representation. Make sure that this representation is consistent with previous ones. You might need to use additional definitions of physical quantities or laws combined with these equations to solve the problem. • Carry out the plan • Solve. Use mathematical relationships from part 2.c to solve for the unknown quantity (quantities). Make sure that you use consistent units. If you do not have enough equations to solve for what you need, go back and check all above steps to make sure you haven’t overlooked some piece of physics given or implied by the situation. • Symbolic and numeric solutions. A complete solution should have the equations given in terms of the symbols, and only then should you plug in numbers to get a numerical answer • Look back – explain what you did, was your answer as expected? • Evaluate the result. Have you answered all parts of the question? Is the number reasonable? Are the units appropriate? Does the result make sense in limiting cases? Include a written explanation for why your result makes sense and what it tells you about the physics of the situation (what happens?) • If solution does not make sense… go back and re-visit your interpretation of the problem and the assumptions you made – did you overlook something? Was something that you thought could be ignored too large to ignore? Check your math, did you make a mistake?

28. Ph212 Problem Solving detail • Understand and restate the problem • Read. • Visualize. • Simplify. • Picture and translate. • Devise and explain the plan • Determine what concepts/laws apply. • Represent physically. • Represent mathematically. • Carry out the plan • Solve. • Symbolic and numeric solutions. • Look back – explain what you did, was your answer as expected? • Evaluate the result. • If solution does not make sense… • (Adapted from ISLE and U. Minn)

29. Ph212 homework Grading Rubric

30. Homework Grading Rubric detail • 4 a. Evaluation of the result • 0:No evaluation is given • 1:Very little information is given to evaluate the result • 2: A partial explanation is given for why the result makes sense (or does not make sense if the incorrect answer was reached), and what it tells us about the physics of the situation • 3: A clear and complete explanation is given for why the result makes sense (or does not make sense if the incorrect answer was reached), and what it tells us about the physics of the situation

31. Guiding Questions for Physics Writing (Paradigms – Junior year) • 1. State the problem. What is the problem that you are trying to solve, and what – if any – assumptions or idealizations are being made about the physical situation. • 2. Outline the general strategy. What physics concepts are relevant? Which general physical equations will be useful in solving this problem? Explain how the physical quantities are related to one another? Connect the dots between any quantities in any ways that you can. • 3. Explain your terminology. What is the role of each of the symbols in these equations? For constants, just list their names and values if used in numerical calculations. For variables, briefly describe what they represent. • 4. Set-up your equations. How did you apply the information in your problem to the general equations? How did your example fit into and change the general equation. Think about how you went about putting in the information from the example you cared about, and any raw data taken, into the general equations. • 5. Explain any data taking procedures used in collecting information needed to solve to solve the physical problem. Remember to include all pertinent information, including how to setup any apparatus used and detailed instructions on how data was acquired. • 6. Organize your data. List any raw data taken. Use graphs and charts to show concisely the relevant quantities in relationship to one another. • 7. Analyze your data. Explain how the data fits into the theory governing the problem you are solving. Comment on any unusual or anomalous data, providing an explanation of how it may have come about being recorded. • 8. What were the mathematical manipulations used in the process of solving the problem? Show the steps of algebra used to solve any tricky parts of the problem, write a short sentence for each explaining why they are true, and include any areas of difficulty that may have lead to dead ends. • 9. Reflect on your final answer. What is it that this answer tells you about the physical quantities involved, and how they are related to each other? Is this a limiting case, or are there limiting cases to this answer for which it is valid? Were there any better ways to solve the problem that you could consider? How did your solution compare and tie into work that others have done in this field of work? What was the most important, significant finding made in solving the problem?

32. Guiding Questions detail 1. State the problem. 2. Outline the general strategy. 3. Explain your terminology 4. Set-up your equations 5. Explain any data taking procedures used in collecting information needed to solve the physical problem. 6. Organize your data. 7. Analyze your data. 8. What were the mathematical manipulations used in the process of solving the problem? 9. Reflect on your final answer.

34. Rubric detail: • Content Criterion: Did the writer convey an understanding of what the final results tell about the physics? • Very Good: Writer clearly explained what the final results tell about the physics of the problem and described what is physically interesting or unique about the solution to the problem. • Fair: An attempt is made to relate the mathematical manipulations to the physical concepts, but the physical situation is weakly related to these results. • Poor: The writer made no attempt at describing how their final solution related to the physical concepts.

35. Improving Efficiency • Problems with current homework system: • High grading load for paid undergrad workers • Papers get lost • Returning papers is a pain • Recording grade takes time and yields errors • Moving to online homework: • Much of the work is graded automatically • Records are kept automatically • Writing-aspects can be built into the existing problems and graded online more efficiently • Gain additional features such as tutorials

36. Curricular Assessment Plans/Ideas: • Concept tests and exemplar problems on exams • Attitude/epistemology surveys • Free response surveys • Specific assessments based on course goals (example assessments from Purdue): • looking at conceptual thinking in problem solving • Interviews using talk-aloud protocol • Are important course ideas/skills are being used by students • looking at TA training and attitudes toward inquiry-based learning • see if the TA attitudes toward teaching and learning are impacted by teaching the course

37. Abstract • Oregon State University has an innovative award wining upper-division physics curriculum, but fairly traditional lower division large introductory courses. Mainly due to staffing constraints, little had been done to improve these courses despite the department’s dedication to team-based curricular development and active-engagement classrooms. More resources were needed to bridge these ideas into the intro courses, leading to my hire charged with leading the introductory course reform efforts. My expertise is in developing quantitative measures for studying the effectiveness of writing to learn (within the context of physics). I will report on ways that writing can be integrated into large introductory courses in a way that scaffolds students toward goals in our upper division courses, without adding a heavy burden on grading. As part of our curricular reform we are also renovating new classroom space: both large lecture hall space and a smaller active-engagement classroom. As part of this planning I recently attended the national Project Kaleidoscope meeting titled “Roundtable on the Future Undergraduate STEM Learning Environment.” I will report on lessons learned at this meeting and our vision for integrating our curricular reform with the classroom remodels.