모바일 추론 / 학습

1. 모바일 추론/학습 연세대학교 컴퓨터과학과 2006. 11. 이영설

2. Prediction, Expectation, and Surprise: Methods, Designs, and Study of a Deployed Traffic Forecasting Service Eric Horvitz, Johnson Apacible, Raman Sarin, Lin Liao

3. Outline • Introduction • JamBayes: A Traffic Forecasting Service • Identifying Key Bottlenecks • Collecting a Case Library and Learning Models • Visualization for Relaying Inferences • Study of Prediction Quality • Learning Models of Competency • Route-Centric Alerting • Reasoning and Alerting about Surprises • Surprise Forecasting • Learning and Using Models of Future Surprises • Accuracy of Models of Future Surprise • Summary and Future Work

4. Introduction • Developing a traffic forecasting service • Monitoring traffic patterns • Predictions about traffic forthcoming congestions • Flowing to users in mobile settings • Used by over 2,500 people now • Predictive Model • Target • the status and dynamics of traffic in Greater Seattle area • Evidence in learning and prediction • Incident report (The Washington Department of Transportation) • the occurrence of major sporting events • weather reports • time of day • calendar information

5. JamBayes: A Traffic Forecasting Service • Streaming Intelligence • Server-based learning and reasoning systems to portable devices • Learning and reasoning based on information from devices and other sources • Smartphlow Web service • Providing users with current status and predictions about the future of traffic flow at key hotspots within the Seattle highway system • Displaying color coded segments on major arteries to relay the speeds and densities of cars • Color : green → yellow → red → black • Using information reported by a network of sensors operated by the Washington Department of Transportation(WDOT) • JamBayes : Predictive component of Smartphlow

6. Identifying Key Bottlenecks • The identification of a set of hotspots (bottleneck) • To focus the attention of modeling and alerting to a set of events and states that people care deeply about • To reduce the parameter space of learning and inference effort • Developing an interactive tool that analyzes a large database of system-wide traffic flow data collected over many months

7. Collecting a Case Library and Learning Models • Collecting a case library

8. Visualization for Relaying Inferences • Lightweight navigation method • Depressing 1-9 dialing keys to access different regions • Depressing joystick to toggle between two levels of zoom • Depressing 0 button to check constructed flyover or troublespots • Visualization • The clock is filled with red proportional to the maximum likelihood time that the congestion will persist before it becomes a flowing thoroughfare

9. Study of Prediction Quality • Accuracy of predictions for time until jams will clear and will form (15 minute tolerance) • 예측한 시간의 15분 이내에 교통 상황이 예측대로 된다면 Success!

10. Learning Models of Competency • Reliability Model • Predict whether a base-level prediction will fail to be accurate • Jam의 지속시간이나 너비 등 적은 수의 변수에만 영향을 받음 • Bottleneck 1 과 11에 대한 기본 모델의 신뢰도 모델 샘플 • 신뢰도 모델이 실시간으로 현재 위치에서 기본 모델의 정확도가 신뢰도 threshold 보다 낮다고 판단하면 시스템은 물음표를 보여주게 된다

11. Route-Centric Alerting • Means for users to set up time-dependent route-based alerting • Deskflow • Provides JamBayes inferences in desktop settings • Allows for configuration of alerting on mobile device • Users receive audio and vibratory alerts when specific criteria are met, based on the time of day

12. Reasoning and Alerting about Surprises • To identify surprises, we compare the output of the marginal models with the real-time states to identify rare flows and congestion • If the likelihood of an observed open flow or congestion at a bottleneck occurs with a probability of 0.10 or less, we mark the situation as a surprising situation

13. Learning and Using Models of Future Surprises

14. Accuracy of Models of Future Surprise • Classification accuracy does not provide a valuable signal as the accuracy is invariably reported as high for the marginal model (which assumes no surprise) given the rarity of surprises

15. Summary and Future Work • The system has been made available within our organization and is now in active use by over 2,500 people • Our ongoing research includes investigating alternate machine learning modeling methods, such as exploring the value of boosting, and also considering extensions that explore other inference and modeling methodologies, including particle filtering, continuous time Bayesian networks, and queue-theoretic techniques.

16. Bayesphone: Precomputation of Context-Sensitive Policies for Inquiry and Action in Mobile Devices Eric Horvitz, Paul Joch, Raman Sarin, Johnson Apacible, and Muru Subramani Microsoft Research, One Microsoft Way

17. Outline • Introduction • Learning Models of Interruptability and Attendance • Models of a User’s Interruptability • Models of Meeting Attendance • Computing Expected Cost of Interruption • Performing Cost-Benefit Analysis in Real Time • Precomputing Ideal Interactions with Users • Bayesphone Desktop and Mobile Applications • Summary

18. Introduction • Inference and decision making with probabilistic user models may be infeasible on portable devices such as cell phones • The methods hinge on the learning of Bayesian user models for predicting whether users will attend meetings on their calendar and the cost of being interrupted by incoming calls should a meeting be attended • The precomputation of ideal decision-theoretic policies from probabilistic user model • The caching of the policies on a cell phone for decision making • Using probabilistic models to guide the handling of telephone calls • The cost of interruption VS. the cost of deferral of an incoming call

19. Learning Models of Interruptability and Attendance • A model that is used to infer a probability distribution over the cost of interruption of users • A model outputs the probability that users will attend meetings that appear on their electronic calendar • 위 2개의 모델로 부터 사용자가 방해받을 때 소요되는 기대값 계산

20. Models of a User’s Interruptability • Predictive models of the cost of interruption from evidence associated with a user’s context • Outlook application, including the time of day and day of week of the meeting, meeting duration, subject, location, organizer, the response status of the user (responded yes, responded as tentative, did not respond, or no response request was made), whether the meeting is recurrent or not, whether the time is marked as busy or free on the user’s calendar, whether the user was required or optional, the number of invitees, the organizational relationships of the invitees to the user, and the role of the user (user was the organizer versus a required or optional invitee) • 수집된 데이터를 통해서 User에게 미팅에 대한 평가를 위한 View를 보여준다. • A study of a model constructed from the same 559 appointments and tested on 100 hold out cases showed a classification accuracy of 0.81 for assigning interruptability

21. Models of Meeting Attendance • The model was trained with the same appointments as were used to train the model for the cost of interruption • The personalized attendance model generates, for previously untagged meetings, the likelihood that users will attend the meetings • A study of the accuracy on 100 cases held out for testing found that attendance was classified at an accuracy of 0.92

22. Computing Expected Cost of Interruption • ECI (expected cost of interruption) • : likelihood that users will attend a meeting, given evidential properties E associated with the meeting, obtained via Outlook appointment properties • : the probability that users will assign a cost Ci to the meeting, where i indexes the meeting as being either in low, medium, or high cost • : the background cost of being interrupted in the default situation S, representing the case where a user does not attend a meeting, as captured by the time of day and day of week

23. Performing Cost-Benefic Analysis in Real Time • A key piece of the decision is the cost of deferring calls from different callers • For such an assessment, we allow users to define groups of callers, based on properties of people, so as to provide a manageable set of classes • The tool allows users to create such organization-related groups as peers, direct reports, manager, position higher-up in the organizational chart, person within organization, and people identified in a user’s list of contacts • Critial associates, close friends

24. Precomputing Ideal Interactions with Users • ECC (expected communication cost) • The cost of deferral and cost of interruption for all incoming calls during a period of time • (mornings, afternoons, evenings, and late night for weekdays and weekends) • t : period • fi : frequency of calls in each caller group i that has a cost of deferral lower than the cost of interruption • fj: the frequency of calls in each caller group j that has a cost of deferral higher than the cost of interruption • cdefer : the cost of deferral of each of these caller classes • cring : the cost of interruption of each of these caller classes

25. Precomputing Ideal Interactions with Users • EVI (the expected value of information) • Decision-theoretic measure of the value of gathering additional information that considers the current uncertainties, the likelihood of different answers to a query for more information, and the ultimate influence of the different answers on ideal policies • Ca : the cost of asking the user before the meeting, just the ECI before the meeting begins • Attending meeting : the expected cost associated with being at the meeting • Not attending : the cost of interruption, the background cost associated with the time of day

26. Bayesphone Desktop and Mobile Applications • Desktop application • Running on Windows XP • Performs inference, cost-benefit analyses, and value of information precomputation of ideal real-time actions and inquiries • Application running on Smartphones • Downloads the precomputed policy file from the desktop via a device synchronization program

27. Summary • We have described a project highlighting the opportunity for precomputing inferences from Bayesian networks and coupling these inferences with cost-benefit policies for fielding policies for action and dialog with users on simple end-point devices like cell phones. • We reviewed the construction of probabilistic models that can infer the expected cost of interruption and the likelihood that users will attend meetings on their calendar • We are also working to extend the evidential considerations beyond meeting properties and time, to include such observations as local sensing of location, motion, and ambient acoustical signals, such as those representing a nearby conversation in progress