SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

Real-Time Network Analytics with Storm: Maximizing Efficiency and Performance

pello avatar
pello
31 diapositives · 436 Views · Jul 18, 2014

Explore the power of real-time network analytics using Storm platform for efficient data ingestion, processing, visualization, and analysis. Learn about key techniques and best practices for optimal performance.

#real-time network analytics with storm: maximizing efficiency and performance #real-time network analytics with storm: maximizing efficiency and performance ppt #powerpoint #presentation
Real-Time Network Analytics with Storm: Maximizing Efficiency and Performance - slide 1
01 / 31Real-Time Network Analytics with Storm: Maximizing Efficiency and Performance
1 / 17

Télécharger la présentation

Real-Time Network Analytics with Storm: Maximizing Efficiency and Performance
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. REAL-TIME NETWORK ANALYTICS WITH STORM Mauricio Vacas Fausto Inestroza Sonali Parthasarathy

  2. The Team Anita Mehrotra Data Scientist MauricioVacas Big Data Architect Krista Schnell Visualization Fausto Inestroza Big Data Architect Sonali Parthasarathy Real-Time Processing Susie Lu Visualization John Akred Product Lead Rick Drushal Engineering Lead

  3. WHY REAL-TIME?

  4. PROCESS Real-Time Data Ingestion Exploratory Analytics Model Prototyping Real-Time Rule Execution Distributed Analytics UNDERSTAND REACT

  5. Accenture Cloud Platform Recommender as a Service Network Analytics Services … Big Data Platform

  6. Drivers consumer devices video usage Issues Operational Costs Understanding service quality degradation Inefficient capacity planning

  7. VISUALIZE STORE PROCESS INGEST ANALYZE

  8. WHY STORM?

  9. What do we need? Multiple use cases Processing, computation, etc. Data types, size, velocity Scalability Mission critical data Fault-tolerance Reliability Time series / pattern analysis

  10. How do we get this from Storm? Processing, computation, etc. Low-level Primitives Scalability Parallelization Fault-tolerance Robust fail-over strategies Processing guarantees Reliability

  11. PRIMITIVES

  12. Tuple Tuple Suboptimal network speed, geospatial analysis Topology Request info (IP, user-agent, etc) Stream Pull messages from distributed queue Spout Sessionization, speed calculation Bolt

  13. PARALLELISM

  14. Supervisor Supervisor W W T T T T W W T T T T Nimbus Zookeeper

  15. Topology Worker Process Executor Executor Task Task Task Task

  16. FAULT TOLERANCE

  17. Supervisor W T T W T T Supervisor W T T Supervisor W T T T T W Nimbus T T W T T T T

  18. RELIABILITY

  19. IP2 IP2 IP1 IP3 A

  20. IP2 IP2 IP1 IP3 A

  21. Suboptimal Network Speed Topology An Example

  22. Tuple (ip 1) Tuple (ip 1) Tuple (ip 1) Tuple (ip 1) Tuple (ip 1) Tuple (ip 1) Tuple (ip 1) Kafka Spout Pre-process Sessionize Calculate N/W Speed per Session Update Speed per IP Identify Suboptimal Speed Store in Cassandra Cassandra

  23. Tuple (ip 2) Tuple (ip 1) Tuple (ip 1) Tuple (ip 2) Tuple (ip 2) Tuple (ip 2) Tuple (ip 1) Tuple (ip 1) Tuple (ip 2) Tuple (ip 1) Tuple (ip 2) Tuple (ip 1) Tuple (ip 1) Tuple (ip 1) Tuple (ip 2) Tuple (ip 2) Tuple (ip 2) Tuple (ip 1) Parallelism Kafka Spout Pre-process Sessionize Calculate N/W Speed per Session Update Speed per IP Identify Suboptimal Speed Store in Cassandra Cassandra

  24. Tuple (ip 1) Tuple (NY) Tuple (ip 1/NY) Tuple (ip 1/NY) Branchingand Joins Kafka Spout Pre-process Sessionize Calculate N/W Speed per Session Update Speed per IP Join Compare Speed Store in Cassandra Stream 1 Cassandra Stream 2 Kafka Spout Speed by Location

  25. Rule Execution

  26. METHOD2 Storm + Drools METHOD1 Storm Drools

  27. Storm + Drools Kafka Spout Pre-process Sessionize Calculate N/W Speed per Session Update Speed per IP Identify Suboptimal Speed Store in Cassandra Drools Cassandra

  28. Integration with Cassandra Cassandra Optimal for time series data Near-linear scalable Low read/write latency Custom Bolt Uses Hector API to access Cassandra Creates dynamic columns per request Stores relevant network data

  29. Lessons Learned • Rebalance Topology • Tweak Parallelism in bolt • Isolation of Topologies • Use TimeUUIDUtils • Log4j level set to INFO by default

  30. DEMO

  31. Next Steps • Trident • Externalizing Rules • Predictive Models • Real-Time Notifications