1 / 22

Real World Activity Recognition with Multiple Goals

Real World Activity Recognition with Multiple Goals. Derek Hao Hu With Sinno Pan, Vincent Zheng, Nathan Liu, Qiang Yang Hong Kong University of Science and Technology Ubicomp 2008 Presentation COEX, Seoul, Korea September 21, 2008. Activity Recognition Applications.

kemal
Télécharger la présentation

Real World Activity Recognition with Multiple Goals

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Real World Activity Recognition with Multiple Goals Derek Hao Hu With Sinno Pan, Vincent Zheng, Nathan Liu, Qiang Yang Hong Kong University of Science and Technology Ubicomp 2008 Presentation COEX, Seoul, Korea September 21, 2008

  2. Activity Recognition Applications Assisting the sick and the disabled. Location-based services, suggest routes via activity / goal recognition Security-related applications

  3. Keywords • Activity Recognition • Input: Sensor readings (may be various kinds) across a number of time slices • Output: The actions / goals inferred at each time slice • Concurrent: several goals are pursued in the same time slice • Interleaving: goals are pursued non-consecutively, in that one goal is paused and then resumed after a while during which time another goal is being pursued.

  4. Examples If you are enjoying your breakfast…(Action 1) Interleaving activities With an egg boiling on the stove… (Action 2) The egg is now boiling! What should you do now? PAUSE Action 1 and Start Action 2 After that, just go back and resume Action 1, with your peeled egg and your unfinished breakfast.

  5. Examples If you…are enjoying your breakfast… (Action 1) Concurrent Activities [SHOULDN’T IT BE LUNCH? OK OK, If you are having your LUNCH] (Action 1) And you are watching TV at THE SAME TIME? (Action 2) Action 1: Having Lunch and Action 2: Watching TV are happening at the same time slice, therefore, it’s…

  6. Questions we plan to answer • In real-world situations, how often do users pursue goals at once in a concurrent and interleaving manner? • If humans often pursue several goals in a concurrent and interleaving manner, are we able to detect such complex social and behavioral patterns from sensors alone? • Do these goals have any deep association with the algorithm complexity of the solutions that are aimed at recognizing the goals?

  7. Psychologists believe… • “Unlike the sequences directed to a single goal in a simple or technical plan, human intended action is influenced by multiple goals.” [Oatley 1992] • The main characteristic of human planning is to reason about problems arising from multiple goals [Wilensky 1983] • Similar claims can be found in educational psychology, cognitive science and anthropology.

  8. How we answer these questions • Three angles • 1. Investigate the dataset, showing that pursuing multiple goals is commonplace in human activities • 2. Propose a solution using Conditional Random Field (CRF) for this multiple goal recognition problem • 3. Analyze the granularity of goal composition graphs, and later show that different accuracies can be achieved in different levels of the hierarchy.

  9. Related Work • There has been many papers related to activity recognition and due to time and space constraints, we can only list a few. • [Patterson et al. 2003] Inferring high-level behavior from low-level sensors • [Patterson et al. 2004] Opportunity knocks: A system to provide cognitive assistance with transportation services • [Intille et al. 2006] Using a live-in laboratory for ubiquitous computing research • [Logan et al. 2007] A long-term evaluation of sensing modalities for activity recognition • [Hodges and Pollack, 2007] The use of electronic sensors for human identification • There are many other related papers in AI conferences, check Henry Kautz, Liao Lin, Jie Yin, Qiang Yang’s paper for more algorithmic solutions. • However, to our best knowledge, no paper before has formally posed out the multiple goal activity recognition problem and try to analyze it with real world datasets as we do.

  10. Analysis of Dataset: Goal Hierarchy • MIT PlaceLabHouse_n PLIA1 Dataset • Recorded on Friday 03/04/2005 from 9am to 12am with a volunteer in PlaceLab • Lowest level of activities are extracted from the original data, including activities such as “sweeping”, “washing-ingredients”, etc. • Relevant activities are combined together into some higher level activities, such as “dealing-with-clothes” • At the highest level, 9 categories of activities are differentiated from each other.

  11. Goal Hierarchy Subfigure 1 (Please check Subfigure 2 and 3 in my homepage: http://www.cse.ust.hk/~derekhh/)

  12. Existence of multiple goals

  13. Goal composition types Our approach plans to extend the activity recognition and the possible goal composition types to all the five possible types here. Previous approaches try to tackle activity recognition / goal recognition problems in these two types only.

  14. Our Proposed Approach: Skip-Chain Conditional Random Field (SCCRF) Intuitively speaking, we are using the long-distance dependencies to capture the relatedness between interleaving activities. Two problems we need to solve: 1) Q: How do we add the “skip edges” between nodes? A: Learn the posterior probability , then set a predefined threshold for setting such “skip edges”. 2) Q: What is the feature function we are using in this CRF model? A: The feature function is highly domain and application dependent. We can refer to Liao Lin and Douglas Vail’s work for some suggestions about some good examples of feature functions in location-based activity recognition. For technical details, please refer to: Derek Hao Hu and Qiang Yang, CIGAR: Concurrent and Interleaving Goal and Activity Recognition, in AAAI 2008.

  15. Our Proposed Approach: minimizing objective function in goal graph Our goal is to learn a similarity matrix between different goals, and use this matrix to minimize an objective function described below. Such an approach is for tackling concurrent goals. This part is set as a “regularizer” such that there will not be too much difference between the tuned probability and the initial probability inferred from the CRF. If the similarity between two goals are large enough, the probability inferred would be small, constrained by the first part. Still two questions we need to answer: Why would a similarity matrix be useful?In real-world examples, it may be more likely that goal A is being pursued if goal B is known to being pursued at the same time, e.g. “eating-dinner” and “sitting-at-table”. Dissimilarity matrix may also help, but we didn’t discuss it here. What is the intuitive explanation of the objective function? For technical details, please refer to: Derek Hao Hu and Qiang Yang, CIGAR: Concurrent and Interleaving Goal and Activity Recognition, in AAAI 2008.

  16. Do Humans Pursue Multiple Goals? (Question 1) A1: YES Proportion of concurrent goals increase as levels go higher. The number is really large ,more than 50% at the highest level As the window size get bigger, concurrent goals are more frequently.

  17. Figure Time

  18. Can we accurately predict multiple goals? (Q2) A2: YES This is our algorithm, compared with the commonly used Naïve Bayes algorithm on a WiFi dataset we had collected in our CSE Department Office. Two parameters are tuned with different values to show the stability of our algorithm.

  19. Differences in goal hierarchies? (Q3) Experiment on the MIT PlaceLab dataset, with accuracies tested on different levels of the goal hierarchy we had constructed, still shows promising improvement over Naïve Bayes.

  20. Another small experiment We designed this experiment since we are curious about the “learnability”, “sensitivity” or “differentiability” of different kinds of sensors. We would like to know, would there be a big change when we use different kinds of sensor readings for training and testing? What would the result be?The experiment is simple and we plan to look into this problem further in our future discussions.

  21. Future Discussions • Some future topics: • 1) Can we do better when there are multiple users? • Multiple-user multiple-goal activity recognition • Distinguish between different users with different actions • Actions may collaborate, compete…as in Wilensky’s description • 2) Can we construct a goal hierarchy automatically from the observation sequences? • Learn which actions are more similar that can be constructed together toward a higher level of action • 3) Can we automatically choose the “best” granularity with a few labeled sensor readings? • Aid the application where a certain accuracy criteria must be met.

  22. Thanks for your attention! Questions?

More Related