Large Scale Assessment in the 1970’s Edward Roeber Michigan State University

Large Scale Assessment in the 1970’s Edward Roeber Michigan State University

Local Assessment in the 70’s • Local districts administered NRTs at virtually every elementary and junior high grade level • Districts tested because ‘good districts should test their students annually….’ • Results used to look at trends in performance • Results not used for instructional improvement • Districts often participated in “state testing programs,” run by the state or a state university

State Assessment in the 70’s • Some states began to assess students with state-selected or state-developed assessment instruments • Originally, individual student instructional improvement was the goal • As scores got publicized, however, school (and then educator) accountability emerged as an assessment purpose

State Assessment in the 70’s • During the 70’s, NAEP was new and there was considerable curiosity about it and its scores • Some states (CT, MN, ME, others) ran “piggyback” assessments in which they administered some or all of the NAEP assessments to state samples of students for comparative purposes • Some states embedded released NAEP items in their assessment programs

State Assessment in the 70’s • During the 1970’s, less than one-half of the states had a state assessment program • Two important trends occurred during this time: • Statewide implementation of CRTs • High school graduation testing • Debra P v. Turlington was the landmark court case/appeal that set the stage for competency testing for high school graduation in almost half of the states today

State Assessment in the 70’s • The predominant form of assessment until the 70’s was norm-referenced tests • In the early 70’s, CTB and Michigan began the creation of an objective-referenced test • This ORT morphed into a CRT since results from an ORT are inherently difficult to interpret • CRTs were condemned by those in the testing industry as an unreliable, invalid, fad that would soon disappear

State Assessment in the 70’s • Criterion-referenced testing spread - aided by the ESEA reauthorizations such as IASA and NCLB • States began to assess, typically, about 3-4 grades, with assessments in math, reading/language arts, and sometimes science and social studies • Scores were reported in terms of percent-proficient, which added accountability pressures on schools

State Assessment in the 70’s • Because of the pressures on states and the new programs, the Association of State Assessment Programs, a virtual organization, was formed in 1977 • Co-chaired by Tom Fisher and Ed Roeber • The group met twice yearly and served as an induction for a number of state assessment staff and states until the late 1990’s

LSAC Conference History • First Conference - 1971 • Sponsored by ECS • Boulder, CO • Engineering College • Attended by about 50-75 persons • Focused on NAEP exclusively • National NAEP assessments • Assessment methodologies • Piggy-back state assessments

Conference History - 1974 • Conference shifted to the Harvest House hotel itself • 100 attendees • Three-day conference • Still focused almost entirely on the NAEP project, but there were a few state sessions • States began to complain about conference emphasis on NAEP

Conference History - Late 1970’s • By late 1970’s, assessment directors still were complaining that the conference focused too much on NAEP • Past attendees became part of the conference planning • Assessment directors became part of the conference session selection

Conference History • The conference (number 0) that did not count • Ann Arbor, MI • 1970 • Focused on NAEP exclusively • SEAs were the only attendees

Conference Sites

Conference Site Criteria (Roeber) • Ideally, site should be located in the Mountain time zone; some sites in the Pacific time zone may also work. • Can participants leave the site at noon on the final day, get to the airport and return to the east coast (e.g., Washington, DC) the same day? • Site should be located near or in mountainous areas. • Big cities are a last choice. • Daytime temperature should not normally exceed 90 degrees.

Conference Site Criteria (Roeber) • Site should be near a major airport, served with jet service non-stop from the major seven airline company’s main hubs • (American: Chicago/Dallas; Continental: Cleveland/Newark/ Houston; Delta: Cincinnati/Dallas/Salt Lake City; Northwest: Detroit/Memphis/Minneapolis; TWA: St. Louis; United: Chicago/Denver; U.S. Air: Pittsburgh/Charlotte/Indianapolis). Service by a discount airline such as Southwest, Kiwi, Western Pacific, and so forth is an added plus. • Six non-stop connections from two or more airline companies (not their commuter affiliates or subsidiaries) is a minimum.

Conference Site Criteria (Roeber) • The site should be no more than a one-hour drive from the airport, with frequent shuttle service (twice an hour minimum) at reasonable cost ($25 per person round trip). • Return service, when many participants may be leaving at the same time, is also important. Hotel(s) • Ideally, all sleeping rooms should be located in one hotel or hotels no more than 2 blocks away from each other.

Conference Site Criteria (Roeber) • The hotel should have space for an opening session (700 persons classroom or luncheon), four-five large-group sessions (200 each classroom), and eight-ten small group sessions (100 persons each classroom) • The vast majority (90%) of rooms should not exceed the Federal rate of $120/night; the rooms should be good quality and quiet. • Catering costs should be reasonable: under $10 (++) for buffet breakfast, under $15 (++) for catered lunch, and under $25 (++) for catered dinner.

Conference Site Criteria (Roeber) • There should be at least two restaurants on-site, and they should offer reasonably priced lunches and dinners; “reasonable” is defined as the Federal travel rate for the city. • Fast service for lunch is a plus. • The hotel should provide good inside and outside fitness opportunities. • Conference registration area should be near the sessions, adjacent to the office. • On-site audio-visual service is a plus. • There should be plenty of space near the opening and large-group sessions for the group to gather and meet informally.

Conference Site Criteria (Roeber) Community • There should be restaurants not connected with the hotel(s) within walking distance of the hotel. • Ideally, some of these should be high quality ones. • There should be one or more interesting places to hold off-site functions to be sponsored by vendors. • There should be high quality/unique places for vendors to take their clients. • There should be interesting sightseeing opportunities for adults and for families in the area of the conference.

Conference Social Activities • In the early years • No conference-wide, company-sponsored social activities • Opening night BBQ on a dude ranch • Good restaurants in Boulder used for vendor dinners • In early 1980’s • First conference-wide social activity was a few bottles of wine and a few bags of chips time after the sessions were over each day (MRC was the leader)

Conference Social Activities • By late 1980’s • Vendors were competing with one another for elegant events • Had to set up a “pecking order” for sponsors • A day was added to the conference so that more events could be held • On some evenings, there were two or more events • Sample events - elegant events, dances, musical groups • No need to buy breakfast, lunch or dinner for entire week • In 1990’s • Companies cut back - response to perceived anti-NRT tenor of the conference • Larger vendor dinners or events for clients only

Innovations Presented as LSAC • The conference has been a place where new ideas, controversial issues, and state innovations have been presented and discussed. For example, in the 1970s: • John Cannell • Rasch • State NAEP • FERPA • Truth in Testing

Topics Covered in the 1970’s • NAEP in general • NAEP exercise development • NAEP secondary research • NAEP operations • Use of NAEP results • NAEP piggyback assessments • Defining mastery levels

Topics Covered in the 1970’s • Evaluating an assessment program • Using and reporting state assessment results • FERPA • Searching for Truth in Truth in Testing • Computer software • Measuring change

Topics Covered in the 1970’s • Teacher competency testing • Student competency testing • Assessment of bilinguals • Assessment and accreditation • Indices of educational effectivness • Affective measurement • Psychomotor assessment

Conference Speakers in 1975 • Alan Morgan (NM) • Jack Schmidt (NAEP and conf. organizer) • John Adams (MN) • Dave Bayless (RTI) • Dick Hulsart (NAEP) • Ina Mullis (NAEP) • Lorrie Shepard (University of Colorado)

Conference Speakers in 1975 • Wayne Martin (NAEP) • Carmen Finley (AIR) • Frank Womer (University of Michigan) • Ed Roeber (Michigan) • Evelyn Brzeszinski (NWREL) • Judy Shoemaker (USED) • Rich Hill (RMC)

Conference Speakers in 1975 • Gerald Bracey (Virginia) • Walt Haney (Huron Institute) • Richard Stiggins (NWREL) • Roy Forbes (NAEP) • James Hertzog (Pennsylvania) • Ron Hambleton (Univ. of Massachusetts) • Tom Fisher (Florida)

Conference Speakers in 1975 • Steve Koffler (New Jersey) • Stan Bernkopf (Georgia) • Ross Green (CTB) • Joe Ryan (University of South Carolina) • David Wright (NAEP) • Jim Impara (Oregon) • Brud Maxcy (Maine)

Reactions from Past Attendees • I greatly valued the small group discussions that characterized the conferences of the early and middle 1970’s. There always was sufficient time for hallway conversations and personal contacts over a cup of coffee. By contrast, recent conferences have been so crowded that it is impossible to participate in all of the relevant sessions, and it is quite likely that one cannot squeeze into the rooms where the most important topics are being discussed. The old intimacy and in-depth discussions have been replaced, and I miss them. • Tom Fisher, 1972

Reactions from Past Attendees • I attended my first conference in 1974 when the “large” in Large Scale only referred to the scope of the testing programs, not the size of the conference. It seems to me that there were only 100 participants back then, and everyone knew one another. Back in the “old days” the conference was always held in Boulder, Colorado. One of the receptions was usually outdoors, and it always seemed to rain just when the bar opened. For many years, CTB hosted the opening reception, although sometimes it was on the second night. These receptions were often theme events—typically related to the Wild West environment. Folks wore cowboy boots, cowboy hats, and bandanas, and the band was usually “country.” • Michael Kean, 1974

Reactions from Past Attendees • This conference has been the one constant activity in my professional career. It has been an annual time to take stock professionally. I have benefited by keeping current on innovative assessment policies and practices, plus interacting with many of those active in the field. On a personal note, I have loved to watch the growth of the conference and how much it means to a number of other attendees. I have special memories of the year in Boulder that I tore cartilage in my knee and had to have surgery on it, as well as the year that it snowed the night before the conference began (while I was camping with family). • Ed Roeber, 1974

Large Scale Assessment in the 1970’s Edward Roeber Michigan State University