Reporting qualitative data

1. Reporting qualitative data

2. The problem: how to convey to the rest of the design team what was learned in qualitative needs assessment?

3. Representation • No representation is an objective, unbiased report of what’s real • “Representations of work [are] interpretations in the service of particular interests and purposes, created by actors specifically positioned with respect to the work represented.” Suchman, Making Work Visible, p. 58 • A representation consists of: • A point of view • Selections and deletions • Foreground and background • Representations embody interpretations • About what is being represented • About how the representation will be used • “The process of representation creates situated artifacts intended to be used in situated ways, in interaction with other people and the world. “ • Representations create as much as they reflect the world.

4. Representation, cont. • For representa tions to be seen as faithful to reality, the work of representation gets “deleted,” which makes invisible the choices made and by whom, and what gets left out. • A key question: Who represents whom or what? • Power • “Self-representation is a form of empowerment.” Suchman

5. Representation, cont. • From introduction to a volume on representation in science: • “If the studies in this volume agree on anything, it is that scientists compose and use particular representations in a contextually organized and contextually sensitive way....The studies in this volume endeavor...to show that the particular ‘representations’ they discuss have little determinate meaning or logical force aside from the complex activities in which they are situated.” Lynch & Woolgar 1990 p. vii-viii.

6. Representation and seeing • Prior to the work of representation is that of seeing. • We learn what to pay attention to and what to see; what is significant and what can be overlooked. • Goodwin shows how archaeology students learn to “see” the color of dirt by specific, hands-on practices of wetting the dirt and comparing it with a standard color chart, • And how an attorney could change how the jury “saw” the videotape of the Rodney King beating through the eyes of police officers. • Representations help teach people how to see. • Law & Lynch 1990 show how the naturalistic drawings and photos in field guides to birds differ in how they choose to present the birds, what factors they emphasize, and how they diverge from pure “naturalism” in order to make apparent the details needed for identification.

7. Invisible work • Work that is take for granted • Work that is only visible when it fails or breaks down • Undervalued • Often left out of representations and therefore out of design • Includes work that is not talked about – doesn’t fit normative view of how work should proceed • Process vs. practice • Failures, problems, difficulties • Includes articulation work: • The work required when things go off-track to bring them back on-track

8. Ethnographic/qualitative investigation, design and usability, and representation

9. Locus of translation from empirical observation to design • Observers develop requirements, formally or informally • The translation from empirical world to requirements is done by the observers, invisible to the system developers • Observers often participate in design work as proxies for users • Observers describe work; design team translates to requirements • Ethnographic narratives: no fixed format • Formalizations • Contextual inquiry: elaborate set of models/diagrams requires team to be familiar with modeling conventions • Other kinds of flow charts, descriptive reports • Observers may participate; or may hand off to development team

10. Scenarios and Personas • Currently very popular; fads • Can be used to summarize and report qualitative findings • Can also be used to perpetuate stereotypes, disempowering views of users

11. Scenarios - Carroll • Stories about people carrying on an activity. • Problem scenario: problem as it exists prior to technology • Design scenario: describes new vision • Consist of (Rossen &Carroll, p. 18): • Setting, context of use • actors with high level goals, task goals, and plans; • actions (observable behavior) • events (external actions produced by computer or other features of a setting) • Evaluation (mental activity interpreting situation) • Plot – sequence of actions & events • People’s goals, plans, and understandings can change. • Suchman, Plans and Situated Actions: rafting example. • Promote a work orientation.

12. Scenarios – Cooper • Daily use scenario: frequent actions • Necessary use scenario: ALL actions that MUST be performed • Edge case scenario: unusual events

13. Benefits • Concrete and flexible – can keep open exactly how ends are achieved. • Tentative working representation that can be elaborated, revised. • Focus on consequences of specific design decisions. • Balance action with reflection – ‘what if.’ • Describe problem situation in a way understandable to all stakeholders. • Focus discussion on ends to be achieved before design details. • Useful for ‘thought experiments’ concerning various design & use possibilities.

14. Personas • Fictional people with life stories, goals, tasks. • Personas come before scenarios. • Instead of focusing on the task in a work context, focus on a whole person (performing a task) • Powerful, engaging – more interesting than scenarios? (Grudin and Pruitt) • A fad that may or may not endure.

15. Where do scenarios & personas come from? • Designers’ ideas, assumptions about how technology could be used and by whom. • Engagement with users: user and task analysis, contextual inquiry, ethnography. • Designers’ and/or users’ hypotheses about possible future activities, users.

16. Uses of scenarios & personas • Problem scenarios & personas: • Develop and illustrate assumptions about users and uses or targeted users and uses • Summarize empirical findings about users, uses • Used • For design, communication within design group • “Configuring the user” • For communication with users (e.g., “is this realistic?”) • For evaluation of designs, prototypes • E.g., walk-throughs • Design scenarios: • Present possible solutions, identify possible problems

17. Benefits of scenarios and personas • Easy way to summarize and make memorable complex info • Easy shorthand for communication • More realistic than abstractions; more closely related to real world (if designed appropriately) • The power of narrative: complex personalities and story lines in an easily-understood format • A way to translate empirical findings into a form that is communicable and usable

18. Risk: substituting fiction for reality • Too much reliance on something not sufficiently grounded empirically. • May seem more empirical, more “true” than they are. • Too few, too simple of scenarios. • Tendency to reduce complexity to one or a few actors, single story line. • Caricatures, stereotypes. • Confirming rather than challenging assumptions. • The real world is messy. • Unrepresentative scenarios, actors. • Good stories rather than representative ones. • Stories that designers think are representative (but aren’t). • Used in place of empirical info on actual tasks, conditions, actors, work practice.

19. Other risks • Use of stereotypes; reinforcing stereotypes • Inessential information that can be misleading or limiting • Jane is a computer engineer who likes to cook and has a 5-year-old daughter; Jim is a computer engineer who spends his spare time re-building cars. • (Unintentionally) developing personas that fit the planned/proposed technology rather than technology that fits the personas