1 / 15

Learning, Recognizing, and Assisting with Activities

Learning, Recognizing, and Assisting with Activities. Tom Dietterich Oregon State University. Assumptions. Goal: Integrated, autonomous, and useful AI systems Must collaborate well with people Must recognize and understand human goals, attentional state, costs of coordination, etc.

rsolt
Télécharger la présentation

Learning, Recognizing, and Assisting with Activities

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Learning, Recognizing, and Assisting with Activities Tom Dietterich Oregon State University

  2. Assumptions • Goal: Integrated, autonomous, and useful AI systems • Must collaborate well with people • Must recognize and understand human goals, attentional state, costs of coordination, etc.

  3. Use Case 1: Edit and return document Person2 Person1 Document1 Send Save Attachment Attach Doc1.doc Message1 SaveAs ReplyTo Doc2.doc • AGENT could: • automatically create TODO item when email arrives • remind user when deadline is near • detect when user has finished editing Doc2.doc and offer to send it back to Person1 • automatically remove TODO item when completed Attach Send

  4. Request to review, includes proposal ID and password open URL web page web page web page web page web page web page web page web page paste ID, password fastlane.nsf.gov paste ID, password fastlane.nsf.gov navigate navigate /www.fastlane.nsf.gov/jsp/homepage/prop_review.jsp /www.fastlane.nsf.gov/jsp/homepage/prop_review.jsp navigate navigate fill out review form submit download service/nsf/proposal.pdf logout print Use Case 2: NSF Proposal Review confirmation page • AGENT could: • automatically create TODO item when email arrives • remind user when deadline is near • automatically login and download & print proposal • automatically login and navigate to “review form” page • automatically remove TODO item when completed

  5. From: beers@sri.com ReplyTo Save Attachment g/darpa/calo/management/arpa/Q3-report.doc Edit using WORD SaveAs Chin Save Attachment: Q3-report-chin.doc g/darpa/calo/management/arpa/Q3-report-template.doc Paste Attach Williams Save Attachment: Q3-report-williams.doc Send Sanchez Save Attachment: Q3-report-sanchez.doc Send reminder Use Case 3: Prepare Quarterly Report Attach • AGENT could offer to • automatically create TODO item when email arrives • automatically save attachment and open it in Word • automatically create outgoing email msg, address it to the correct recipients, and attach the template • automatically track the email replies and save the attachments (with the right names) in the right folder • automatically offer to send reminders to the missing subcontractors • automatically open up the template and all replies in Word • automatically attach the final file to a reply email to Melissa • automatically delete TODO item when complete

  6. Research Challenges • Representing Workflows • Learning Workflows • Recognizing Workflows • Deciding (Learning) When and How to Help

  7. Representing Workflows • For what purpose: • execution: • sequence of actions (possibly with conditionals and iteration) • recognition: • partially-ordered sequence of actions (with conditionals and iteration) • capture additional features to aid recognition (e.g., email speech acts) • learning: • need action models to detect unobserved steps and understand goals • assistance • need action models to understand goals • Workflow steps: • informational inputs (file name, file, URL) • action (click Download) • action models (creates file on disk with file name; contents = contents of URL file)

  8. commentOnDocument :- mailArrived(EmailRID, Requester, SpeechAct, Deadline, [Attachment1]), outlookOpen(EmailRID), attachmentSave(EmailRID, Attachment1, FileRID), wordEditDocument(FileRID, EditedFileRID), outlookOpen(EmailRID), outlookComposeReply(NewEmailRID, EmailRID), outlookSendReply(NewEmailRID, Requester, SpeechAct2, [Attachment2]), outlookAttachmentInfo(NewEmailRID, EditedFileRID, Attachment2). wordEditDocument(FileRID, FinalRID) :- wordOpen(FileRID), finishEdit(FileRID, FinalRID). // simply close the file and return it finishEdit(FileRID, FileRID) :- wordClose(FileRID). // close the file, then later re-open it and continue finishEdit(FileRID, FinalRID) :- wordClose(FileRID), wordOpen(FileRID), finishEdit(FileRID, FinalRID). // perform a SaveAs and then continue finishEdit(FileRID, FinalRID) :- wordSaveAs(FileRID, NewFileRID), finishEdit(NewFileRID, FinalRID). Representing Workflows

  9. Learning Workflows • Learning by Demonstration • LAPDOG: PBD system at SRI • Lau, et al. at IBM and before that UW • PLOW: Allen et al. Rochester • Learning by Observation (unsupervised) • Weld et al.

  10. Recognizing Workflows • Challenges on the desktop • Multiple workflows interleaved • Multiple instances of the same workflow interleaved • reviewing multiple NSF proposals • Sharing across workflows • log in and navigate only once, then download multiple files • Unmodeled background events • IM, nytimes.com, weather.com, etc.

  11. Recognition Task • Given: • a set of workflow schemas • an observation sequence • Find: • all instances of those workflow schemas in the observation sequence • detect each instance as early as possible • report the current state of all active workflow schemas at each point in time • Metrics: • false positives, false negatives, timeliness

  12. Assistance • What steps can the AGENT do? • What steps should the AGENT do? • How and when should the AGENT coordinate with the user? • Decision-theoretic collaboration • model the user’s intentions and attentional state • estimate the expected benefit of AGENT’s assistive plan (including coordination cost) • choose action that maximizes expected benefit

  13. Rich Intention Structures • Goal stack • traditional programming languages • hierarchical reinforcement learning formalisms • cognitive architectures: SOAR, ACT-R • Goal graph • ABL (Mateas) • The user’s TODO list is an intention structure • so is the Inbox for many people • Revised statement of our goal: • representation, learning, recognition, and assistance with rich intention structures

  14. Related Topics • Argumentation and Persuasion • How do two agents exchange information in order to reach agreement? • Explanation-based Teaching and Learning • AGENT makes a mistake • user says “Why did you do that?” • AGENT explains • user corrects parts of the explanation • etc. • Transfer Learning • How do I transfer to you something I’ve learned when • you have a different ontology • I can’t give you all of my training data (privacy, bandwidth)?

  15. Summary • Goal: AI AGENT that can help humans • Prerequisite: AGENT must understand what its user is doing

More Related