320 likes | 438 Vues
Composite Subscriptions in Content-based Pub/Sub Systems. Guoli Li. Agenda. Motivation PADRES System Composite Subscriptions Case Study: Workflow Management Evaluations Conclusions and Future Work. Motivation. A large number of atomic events Lack of event correlation
E N D
Composite Subscriptions in Content-based Pub/Sub Systems Guoli Li
Agenda • Motivation • PADRES System • Composite Subscriptions • Case Study: Workflow Management • Evaluations • Conclusions and Future Work Middleware 2005 Grenoble France
Motivation • A large number of atomic events • Lack of event correlation • Potential Applications: • Workflow processing and business process execution • Network management • … Middleware 2005 Grenoble France
The PADRES System • PADRES: Publish/Subscribe Applied to Distributed REsource Scheduling • Features: • Predicate-based subscription language • (attribute, op, value) pair • Extended to supporting composite subscriptions • Rule-based publication/subscription matching • Publications are mapped to facts • Subscriptions are mapped to rules • Advertisement-based routing Middleware 2005 Grenoble France
Content-based Routing • Advertising Advertisement Distributed Overlay Broker Network … … PADRES Middleware 2005 Grenoble France
Content-based Routing • Subscribing Subscription Distributed Overlay Broker Network … … PADRES Middleware 2005 Grenoble France
Content-based Routing • Publishing Publication Distributed Overlay Broker Network … … PADRES Middleware 2005 Grenoble France
Several Concepts • Message • Advertisement • Subscription • Publication • Event • Atomic event: publication in pub/sub literature • Composite event: a pattern of events • Subscriptions • Atomic subscription: atomic event • Composite subscription: composite event Middleware 2005 Grenoble France
& S5 & || || S1 S2 S3 S4 Composite Subscriptions • Composite subscription consists of primitive subscriptions • Provide a higher level view for subscribers • Save network bandwidth • Subscription language features • Operators: AND (&), OR (||) • Variables: ($x) • Advanced event patterns CS={{S1 OR S2} AND{S3 OR S4} AND S5} Middleware 2005 Grenoble France
Composite Subscriptions • Parallelization pattern: s1 & s2 • Alternation pattern: s1 || s2 • Sequence pattern: (s1;[timespan:ts]s2)[within:wi] • Repetition pattern: Repetition(S, n, attr, v) e.g. {rule ((job (appl = $X) (name = A) (state = succ)) & (job (appl = $X) (name = B) (state = succ))) => (forward a notification to proper destination)} Middleware 2005 Grenoble France
Composite Subscriptions • Parallelization pattern: s1 & s2 • Alternation pattern: s1 || s2 • Sequence pattern: (s1;[timespan:ts]s2)[within:wi] • Repetition pattern: Repetition(S, n, attr, v) e.g. {rule ((job …. (name = A) (time = $Y)) & (job …. (name = B) (time > $Y+ts) (time < $Y+wi))) => (forward a notification to proper destination)} e.g. {rule ((job …. (name = A) (attr = $Z)) & (job …. (name = A) (attr = $Z+v)) & … & (job …. (name = A) (attr = $Z+(n-1)v)) ) => (forward a notification to proper destination)} Middleware 2005 Grenoble France
& || S3 S1 S2 Composite Subscription Routing CS={{S1 AND S2} ANDS3} P Publishers Subscribers CS’={S1 AND S2} S P1 Distributed Overlay Broker Network CS B1 B5 S S1 B3 CS’ B4 S2 S3 B2 B6 P2 P3 Middleware 2005 Grenoble France
AND AND S3 S1 S2 Composite Event Detection CS={{S1 AND S2} ANDS3} P Publishers Subscribers S CS’={S1 AND S2} P1 P1 Distributed Overlay Broker Network S1 CS B1 B5 S CS’ CS P12 B3 B4 P123 B2 B6 P2 S2 P2 S3 P3 P3 Middleware 2005 Grenoble France
Advantages • Rule-based approach naturally enables the composite subscription • Event detection results can be shared • Network traffic is reduced • Composite subscription is forwarded into the network as far as possible before it is split • Composite events are detected close to the data source and not widely disseminated Middleware 2005 Grenoble France
Case Study: Workflow Management • Decentralized Workflow Management based on Pub/Sub • Functions • Workflow transform • Workflow deployment • Workflow execution • Workflow control and monitoring Middleware 2005 Grenoble France
Architecture Middleware 2005 Grenoble France
E F Workflow Descriptions • Workflows are described using XML-based languages, such as BPEL • Activities defined in a workflow have dependencies among each other • A workflow instance is generated by a trigger • Failures can be handled in the workflow description trigger A B C D Middleware 2005 Grenoble France
Workflow Transform • The first job(s) subscribe to trigger messages e.g. Job A subscribes to trigger publication: {[class,eq,trigger],[workflow,eq,payroll], [instanceID,=,$x]} • Job dependencies are implied by subscriptions e.g. Job E subscribes to job A failed: {[class,eq,job_info],[workflow,eq,payroll], [nstanceID,=,$x], [job,eq,A], [status,eq,fail]} e.g. Job D subscribes to B and C finished, which is a composite subscription • Advertisements are generated from a workflow specification file Middleware 2005 Grenoble France
BPEL A B C D Workflow Deployment sub/advs Job D sub/advs Job C sub/advs Job B sub/advs Job A Deployer sub/advs Job D Distributed Overlay Broker Network Job D Agent B1 B5 B3 B4 sub/advs Job A B2 Job A Agent B6 sub/advs Job C sub/advs Job B Job C Agent Job B Agent Middleware 2005 Grenoble France
BPEL A B C D Workflow Execution (Success) trigger Workflow Manager END Distributed Overlay Broker Network Job D Agent B1 B5 BC B3 A B4 A sub/advs Job D B2 Job A Agent B6 A sub/advs Job A Job C Agent sub/advs Job C Job B Agent B C sub/advs Job B Middleware 2005 Grenoble France
BPEL A E B C F D Workflow Execution (Failure) trigger Workflow Manager sub/advs Job D Distributed Overlay Broker Network Job D Agent B1 B5 B3 B4 END B2 Job A Agent B6 Job F Agent A Job C Agent sub/advs Job F sub/advs Job A Job B Agent E Job E Agent sub/advs Job E sub/advs Job C sub/advs Job B Middleware 2005 Grenoble France
Workflow Monitoring • Monitor is a Pub/Sub client • Monitor/control workflow execution status • Workflow level • Activity level • Support dynamic workflow modification • Visualize topology of broker overlay network • Visualize message routing Middleware 2005 Grenoble France
Evaluations • Environment • Intel Xeon 3GHz, 1GB RAM • JDK 1.4.2 • Workload description • Predicates based on 20 attributes with random operators • Values are uniformly distributed in the value range • Two workflows with different number of jobs • Metrics • Matching time • Routing delay per composite subscription • Network traffic Middleware 2005 Grenoble France
Publication Matching Time Middleware 2005 Grenoble France
Composite Event Detection Middleware 2005 Grenoble France
Routing Delay Middleware 2005 Grenoble France
Number of Notifications Middleware 2005 Grenoble France
Workflow Traffic Middleware 2005 Grenoble France
Related Work • P.R. Pietzuch, B. Shand, and J. Bacon. Composite event detection as a generic middleware extension. IEEE Network Magazine, Special Issue on Middleware Technologies for Future Communication Networks, Jan 2004 • SIENA • REBECA Middleware 2005 Grenoble France
Conclusions • An expressive subscription language • A rule-based approach of message matching and composite event detection • Case study: a decentralized workflow management system based on PADRES • Future Works • Support BPEL4WS • QoS based workflow execution Middleware 2005 Grenoble France
Questions? Middleware 2005 Grenoble France