1 / 26

User studies

User studies. Why user studies?. How do we know security and privacy solutions are really usable? Have to observe users! you may be surprised by what users really do you are not your users. Typical Security Evaluation. Does indicator behave correctly when not under attack?

grant-good
Télécharger la présentation

User studies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. User studies

  2. Why user studies? How do we know security and privacy solutions are really usable? Have to observe users! • you may be surprised by what users really do • you are not your users

  3. Typical Security Evaluation • Does indicator behave correctly when not under attack? • No false positives or false negatives • Does it behave correctly when under attack? • Can it be spoofed or obscured? Correctindicator Wrongindicator Attacker redirects

  4. Usability evaluation questions • Do users notice it? • “What lock icon?”

  5. IE6 cookie flag Firefox SSL icon Do users know what it means? Netscape SSL icons Cookie flag

  6. Do users know what to do when they see it?

  7. Other usability questions • Are they motivated to take action? • And do they actually do it? • How about over the long term?

  8. Why Johnny Can’t Encrypt • Whitten and Tygar, 1999 • A Usability Evaluation of PGP 5.0 • Pretty Good Privacy • Software for encrypting and signing data • Plug-in provides “easy” use with email clients • Modern GUI, well designed by most standards

  9. Evaluation Methodology • Motivation: Security software may require additional usability considerations • Question: Is PGP usable by everyday users? • Method: Cognitive Walkthrough + User Study • Goal: demonstrate usability problems Question: is method appropriate?

  10. Defining usable security software • Security software is usable if the people who are expected to use it: • are reliably made aware of the security tasks they need to perform. • are able to figure out how to successfully perform those tasks • don't make dangerous errors • are sufficiently comfortable with the interface to continue using it.

  11. The studies • Cognitive Walkthrough: • Tasks: encrypting and signing email, decrypting, etc. • User Study • PGP 5.0 with Eudora • 12 participants all with at least some college and none with advanced knowledge of encryption • Participants were given a scenario with tasks to complete within 90 min • Tasks built on each other • Participants could ask some questions through email

  12. Cognitive Walkthrough results • Visual metaphors • Public vs. Private keys • Signatures and verification • Key server • Hidden? What is it doing? • Revocation not automatic • Several irreversible actions • Can cause serious errors • Consistency • Too much information • More unneeded confusion

  13. User Study Results • 3 users accidentally sent the message in clear text • 7 users used their public key to encrypt and only 2 of the 7 figured out how to correct the problem • Only 2 users were able to decrypt without problems • Only 1 user figured out how to deal with RSA keys correctly. • A total of 3 users were able to successfully complete the basic process of sending and receiving encrypted emails. • One user was not able to encrypt at all

  14. Conclusion • None of their defined usability goals were met. Question: Is this a failure in the design of the PGP 5.0 interface or is it a function of the problem of traditional usable design vs. design for usable secure systems?

  15. Kazaa File Sharing Study • Motivation: Lots of people use P2P file sharing • Problem: Seems like lots of people sharing files accidentally. Why? • Method • Cognitive walkthrough • User study • 12 users, 10 had used file sharing before • Questionnaire for file sharing understanding • Task: figure out what files are being shared by Kazaa (Answer: Download files set to C:\ so all files on the C:\ drive)

  16. Their usability criteria • Peer-to-peer file sharing software is safe and usable if users: • Are clearly made aware of what files are being offered for others to download • Are able to determine how to share and stop sharing files successfully • Do not make dangerous errors that can lead to unintentionally sharing private files • Are comfortable with what is being shared with others and confident that the system is handling this correctly

  17. Cognitive Walkthrough Results • Multiple names for similar things • My Shared Folder, My Media , My Kazaa, Folder for downloaded files • Downloaded files are also shared • Kazaa recursively shares sub-folders • Easy to add directories to share, difficult to remove

  18. User Study Results • 5 people thought it was “My Shared Folder” • which one UI did suggest • 2 people used Find Files to find all shared files • This UI had no files checked, thus no files shared? • 2 people used help, said “My Shared Folder” • 1 person couldn’t figure it out at all • Only 2 people got it right

  19. Generalizing results • Design suggestions: • Only allow sharing of multimedia files • Better feedforward • Allow exceptions to recursively shared folders

  20. A very different study • Motivation: Online social networking widespread. • Problem: People sharing large amounts of personal information, which puts them at risk for variety of problems • Questions: • how and why do users share and protect their information? • how do they form impressions of other profiles? • Goal: Identify requirements, issues and challenges in improving privacy in online communities

  21. Method • User study of Facebook.com • 16 college participants from psych pool • Logged into own profile, information and privacy settings noted • Interviewed about their own profile: • their motivations for entering information, how they formed their social networks, their concerns over others viewing their profiles, etc. • View 4 other profiles, and interviewed about impressions

  22. Big picture results • Users thinking about their privacy mainly during initial activation – while filling out initial profile information • Neglect privacy implications of later interactions – interacting with friends, not thinking about the broader audience at that point… • …Until a negative experience occurs • Need new mechanisms to increase awareness of the accessibility of their profile and their risks – especially during everyday activities. • Need new ways to more easily adjust privacy settings during those everyday activities.

  23. Summing it Up • Examples of how to run user studies • Not the most rigorous studies, but good enough to demonstrate main point • Tradeoffs of various methods? • How to choose methods?

  24. Your Observations • Where did you observe? • What were some general observations? • What problems did people have? • Any privacy or security implications? • What did you think of being an observer?

  25. Now let’s practice • 4 groups: • Voice recorder • Camera • PDA (2) • Take a few minutes to design a simple user study • What questions to you have? • What are your usability goals? • What methods? • Use one member as tester if you can

  26. User Test • Results!?

More Related