160 likes | 237 Vues
E N D
Research • Much research in usability is more-or-less qualitative • Observations • Focus groups • Monitoring systems MAY apply a metric to complaint and act once a threshold is reached. • Still, there may be reasons for gathering quantitative information • what capacity is required • Storage • Simultaneous respons • speeds
Panels • Recruit a panel of people • Answer questions and/or • Be willing to be monitored on actions • Often, open-ended recruiting and/but determine critical demographics • Age • Gender • Location • Device • ? • Do need to decide if those who volunteer are different from the regular population.
Interpret findings • Assume you have accurate model of the user population • Adjust (normalize) findings
Very quick Statistics • Mean • Median • Standard Deviation and Variance • Normal distribution
Sampling • Done to make an informed estimate of something for a large population (of people or things) when it is too expensive or difficult to ask every person or measure every thing. • Typical finding:We are 95% that the actual value or proportion is within a certain rangex- Margin_of_Error <= x <= x+Margin_of_Error
Example • Find out how many people think the latest version of your program is better than the last. • Ask N people. Say p is the proportion that said yes. Margin_of_error = ztransform * square_root((p)* (1-p)/N) Where ztransform is based on confidence level1.96 for 95%.
Example continued • N is 1500. • p is 822/1500 or 54.8% • M = 1.96 * SQRT((822/1500)*(678/1500)/1500) • M is 2.5% • So we are 95% confident that between 54.8-2.5 which is (about) 52.3 % and54.8+2.5 which is (about) 57.3 % think the new system is better….
Caution • There is a chance (say 1/20) that the prediction is wrong. If you want something less, then choose a different confidence level with a different z-transform • Typical choice: 99% confidence, multiply by 2.58Bigger margin means more confident. • We are
Example with different confidence level • N is 1500. • p is 822/1500 or 54.8% • M = 2.58 * SQRT((822/1500)*(678/1500)/1500) • M is 3.3% • So we are 99% confident that between 54.8-3.3 which is (about) 51.5 % and54.8+3.3 which is (about) 58.1 % think the new system is better….
Warning • The formula works if the sample is truly random, that is • Every person in the whole population stands the same chance as being in the sample. • Predictions fail when sample isn't random. • Well-done analysis of election polling works • Reference Nate Silver
Reference • The Cartoon Guide to Statistics by Larry Gonick and Woollcott Smith.
Panels and/or testing • When testing for usability, need to evaluate costs/benefits of formal testing versus access to subjects that will supply more information. • Comments?
Aside: Next year's courses • Fall: Data Structures (including structures for music) • Fall: Topics in Advanced Computing • Spring: Creating Databases for Web Applications • MAYBE: Fall: Networking and Security Spring: Advanced Security • Consider also taking Introduction to Statistics