1 / 14

Intelligent Event Processing (IEP)

Intelligent Event Processing (IEP). Aggregation. Filtering. Partition. Correlation. Aggregation. Time-based Aggregator. Tuple-based Aggregator. Window Aggregator. Time-based Aggregator. Compute statistical summery over a fixed time period.

ajo
Télécharger la présentation

Intelligent Event Processing (IEP)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Intelligent Event Processing (IEP) Aggregation Filtering Partition Correlation

  2. Aggregation Time-based Aggregator Tuple-based Aggregator Window Aggregator

  3. Time-based Aggregator Compute statistical summery over a fixed time period. • Statistics: sum, average, median, lower-quantile, upper-quantile, min, max, standard-deviation, sum-of-square, outlier-percentage. • Example: Given a stream of transactions of a stock, computes a new stream that holds hourly min, average, and max of the stock price.

  4. Tuple-based Aggregator Compute statistical summery over a fixed number of events. • Statistics: same as time-based aggregator. • Example: Given a stream of transactions of a stock, computes a new stream that holds the min, average, and max of the stock price of every 10 transactions.

  5. Window Aggregator Compute statistical summery over a sliding window • Statistics: same as time-based aggregator. • Example: Given the latest 2 hour window of transactions of a stock, computes the min, average, and max of the stock price of those transactions. As time passes by, the statistical summery will change too.

  6. Filtering Standard Filtering Filtering by Point Estimation Filtering by Hypothesis Testing

  7. Standard Filtering • Example: Given a stream of transactions of a stock, computes a new stream that holds only those transactions of which the stock price is > $50.

  8. Filtering by Point Estimation Analyze historical data, and build a Statistical model. Create a filter that filters out only those events that are considered outliers by the statistical model. • Example: Given enough historical measurements of human body: (weight (kg), height (cm)), one build a Statistical model: weight ~ -105 + height + N(0, 10). Given a stream of measurement data, computes a new stream that holds those measurement of which weight is outside of (-105 + height - 20, -105 + height + 20)

  9. Filtering by Hypothesis Testing Analyze historical data and make up a hypothesis. Creates a filter that filters out only those groups of events that failed the hypothesis. • Example: Given enough historical measurements of products produced by an assembly line, one found that the defect rate is 2%. Make up a hypothesis: defect rate <= 2% Given a stream of measurement data, computes a new stream that holds alert events. Test the hypothesis for every 100 measurements. If the defect rate is Statistically greater than 2%, generate an alert event.

  10. Partition Standard Partition Partition by Classification

  11. Standard Partition Example: Given a stream of stock transactions, computes a window that holds the latest 10 transactions for each stock symbol.

  12. Partition by Classification Analyze historical data and build a Statistical classification model. Creates a partition window that holds the latest N events for each classification group.

  13. Correlation Relational Join of Multiple Windows

  14. Relational Join of Multiple Windows Example: Given the latest 2 hour window of stock transactions, and the latest 2 hour window of news, computes the latest 2 hour window of insider-trades by joining the trader’s name with the name mentioned in the news.

More Related