1 / 15

Soylent

Soylent. A Word Processor with a Crowd Inside. Kyong Hwa Bae. Outline. What is Soylent ? Features of Soylent Techniques for making Soylent Evaluation Discussion. What’s Soylent ?. A word processing interface that utilizes crowd contributions

barr
Télécharger la présentation

Soylent

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Soylent A Word Processor with a Crowd Inside Kyong Hwa Bae

  2. Outline • What is Soylent? • Features of Soylent • Techniques for making Soylent • Evaluation • Discussion

  3. What’s Soylent? • A word processing interface that utilizes crowd contributions • To aid writing tasks – error prevention, paragraph shortening, and automation of tasks like citation searches and tense changes

  4. Soylent

  5. Features • Shortn : Text Shortening • Crowdproof : Crowdsourced Proofreading • The Human Macro : Natural Language Crowd Scripting

  6. Challenges in Programming with Crowd Workers • High Variance of Effort • Lazy Turker • Eager Turker • Turkers Introduce Errors 30% error the raw results from Turkers!

  7. The theme of loneliness features throughout many scenes in Of Mice and Men and is often the dominant theme of sections during this story. This theme occurs during many circumstances but is not present from start to finish. In my mind for a theme to be pervasive is must be present during every element of the story. There are many themes that are present most of the way through such as sacrifice, friendship and comradeship. But in my opinion there is only one theme that is present from beginning to end, this theme is pursuit of dreams The theme of loneliness features throughout many scenes in Of Mice and Men and is often the principal, significant, primary, preeminent, prevailing, foremost, essential, crucial, vital, criticaltheme of sections during this story

  8. Techniques for Programming Crowds • Find-Fix-Verify Pattern • Find : to identify patches that need to be edited • Fix : to revise an identified patch • Verify : to control the quality of revisions • Find-Fix-Verify in Soylent • Shortn and Crowdproof use the pattern.

  9. Find-Fix-Verity Word Mechanical Turk Find “Identify” Fix “Edit” Verify “Choose”

  10. Shortn Evaluation • Goal : to see how much Shortn could shorten text, as well as its cost and time • Results : • 78% ~ 90% of the original document length (50% after 3 iterations) • 18.5 minutes of wait time + 2 minutes of work time • $1.41 per paragraph

  11. Shortn Evaluation • Issues : • Shortn occasionally introduces errors. (ex) removing the academic phrase “In this paper we argue that” => stylistically incorrect, capitalization and punctuation problems • Parallelism makes errors. (ex) “the low order bits of the key and next..(the key fragment)” => unmatched cuts

  12. Crowdproof Evaluation • Results : • 88% of error fixing ( 67% error caught from Soylent, 30% from Microsoft Word ) • 18 minutes of wait time + 2.8 minutes of work time • $3.40 per paragraph • Issues : • Lazy Turkers don’t fix all errors. • A lack of domain knowledge (ex) GUIs => GUI

  13. Human Macro Evaluation • Goals : • Intension – Did Turkers understand a task? • Accuracy – Was the result flawless? • Results : • 88% of intension success ( typical errors occur when ask two requirements ) • 70.8% of accuracy success ( failed on some detail or too much information from Eager Beavers )

  14. Discussion • Wait time • Cost • Privacy • Legal ownership • Domain knowledge

  15. END Kyong Hwa Bae

More Related