1. Group Index Assignments Create the items and index for the quality of a professor’s lecture Create the items and index for the quality of an MPA student’s presentation

2. Two Groups, Same Process • Each person list two items (factors) that would determine the quality of a professor’s lecture or an MPA student’s presentation • Pick a moderator • Moderator writes items down one by one using group input to combine responses where appropriate

3. Likert Scaling • Each person write down the items vertically on a piece of paper • Create a horizontal scale from 1-5 • Rate each item according to the scale • Moderator write each item horizontally on board • Moderator write down each score horizontally, moving to the right • Moderator total scores for each item • Eliminate all but top six

4. Relative Importance of Items • Moderator lead team in discussion of relative importance of the remaining items • Is the existing ranking appropriate? Are they of equal import? Do one or two seem more important? If so, how much more? • After the discussion each person write down remaining items vertically on paper • Next to each item write a number between 0-100 (all items need to sum to 100)

5. Relative Importance of Items • Moderator write items vertically on board • Moderator write scores from each person for each item • Moderator (with help) average score for each item • Moderator total up all average score (won’t be 100) • Moderator (with more help) divide each item’s average score by the total (the results added up will be 100%)

6. Report • Moderator (or other duly appointed representative) of each group report results

7. Questions • How did we define the concept to be measured? • Any problems? • How else could we have done it? • How did we select the items? • Any problems? • How else could we have done it? • What type of Validity are we messing with here? • How did we combine the items? • Any problems? • How else could we have done it? • What type of Validity are we messing with here?