1 / 33

Microsoft Azure Event Hub – IOT at Scale

Microsoft Azure Event Hub – IOT at Scale. Brent Stineman – Technical Evangelist and code monkey brestin@microsoft.com. What is the Internet of things? Or Connected Devices if you’re one of the cool kids. Defining the undefinable.

fauve
Télécharger la présentation

Microsoft Azure Event Hub – IOT at Scale

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Microsoft Azure Event Hub – IOT at Scale Brent Stineman – Technical Evangelist and code monkey brestin@microsoft.com

  2. What is the Internet of things?Or Connected Devices if you’re one of the cool kids.

  3. Defining the undefinable The Thing a specialized, autonomous piece of hardware capable of performing a simple, focused task but does not have a ‘traditional” user interface. Data Isolated elements sent or received by the thing Connectivity Able to connect to and interact with another thing or device Management Having an identity we can use to manage the thing

  4. Canonical Arch for processing events Producers Collection Ingestor(broker) Transformation Long term storage Presentation and action Scalableevent Broker Azure Storage Real-time analytics(Storm/Orleans, etc…) Cloud Gateways(WebAPIs) Web/thickclientdashboards Applications - HD Insight Externaldata sources Your are here Massively Parallel Azure Systems w/ Jason Young Tuesday@10:30am, Aralia Devices Search and query HDInsight and Internet of Things w/ Tony Guidici Tuesday, 2:30pm, Tamarind Field Gateways Batching/storageadapters Data analytics (Excel)

  5. Brokered MessagingOne message at a time.

  6. Queues Guaranteed FIFO, with push-style API, up to 80GB maximum size, 10,000 queues per namespace, 2,000 messages per second, 256KB per message.

  7. Queues – increasing throughput • Express • Decreases latency by storing the messages in memory • Partitions • Managed scale out (at max 16x)

  8. Topics A higher level abstraction from queues, enables pub/sub models with similar scale out capabilities and consistent pricing

  9. Topics • Filters • Allows subscribers to selectively receive messages • Rules • Aid in filtering and authorization • Volatile Subscribers • Can delete on idle, allowing for transient consumers

  10. Introducing Event HubWhen the plumbing gets complicated, you just need to go with the flow

  11. What do you need to handle this? An ingestion service that can Support variety (> million concurrent devices) Support velocity (> million events per second) Support volume (> 100s of Tb of event processing) With Buffering to handle variability Durability Low latency Security And be affordable!

  12. Event Hub – IOT at Scale Storage & Analytics Custom Code & 3rd Party Services Event Hub Web/Mobile User Interfaces - Hyper Scale - - Fully Managed - - Interoperable - - Secure - - Cost Effective - Integration Services Cloud Services Event Sources

  13. Event Hub Publisher concept Every Event Hub has pre-defined number of partitions Every event has a partition key => mapped to one of the partitions Partition 1 Event 1 PartitionKey=A Partition 2 Event 1 PartitionKey=B Partition “n”

  14. Getting Started Creating our first hub, sample code

  15. Publish Message via .NET/AMQP Critical to scaling

  16. Publish Message via HTTP http://aka.ms/thatconfEH

  17. Publisher Security <protocol>://<namespace>.servicebus.windows.net/<eventhubname>/publisher/<partitionkey>/messages SharedAccessSignaturesr=https%3a%2f%2fthatconference-ns.servicebus.windows.net%2fthatconference%2fpublishers%2fbstineman%2fmessages&sig=a3MkKnkhMZpTMz%2fnFcsCS%2fcT9rT7Wn%2fZO%2b7eztBRMkk%3d&se=64948904112&skn=Publish

  18. Real World Example Publishing Social Media and Vendor Check-ins

  19. Direct Receiver • Receiver connects to a partition and pull messages • Limited(5) receivers for each partition Worker “n” Worker 1 Receiver 6 Receiver 1 Receiver 2 Receiver “n” Partition 1 Event 1 Pkey = A Partition 2 Event 2 Pkey = B Partition “n”

  20. IEventProcessor– Managed Receivers http://www.nuget.org/packages/Microsoft.Azure.ServiceBus.EventProcessorHost/0.3.0

  21. IEventProcessor– Managed Receivers • Register a class that implements an EventProcessor interface • Client SDK will manage balancing receivers across workers Worker 4 Worker “n” Worker 1 Worker 2 Callback for prtn. 1 Callback for prtn. “n” Callback for prtn. 6 Callback for prtn. 1 Callback for prtn. 2 Callback “n” Partition 1 Partition 2 Partition “n” http://www.nuget.org/packages/Microsoft.Azure.ServiceBus.EventProcessorHost/0.3.0

  22. Pub-Sub with Event Hub • Consumer groups allow multiple, parallel sets of subscribers to receive the same events Worker “n” Worker “n” Worker “n” Worker 1 Worker 1 Worker 1 Consumer Group A Callback for prtn. 1 Callback for prtn. 1 Callback for prtn. 6 Callback for prtn. 1 Callback for prtn. 6 Callback for prtn. 6 Consumer Group C Consumer Group B Callback for prtn. 2 Callback for prtn. 2 Callback “n” Callback “n” Callback for prtn. 2 Callback “n” Partition 1 Partition 2 Partition “n”

  23. IEventProcessor– Partition Management

  24. IEventProcessor– Partition Management

  25. Real World Example Proximity Sensors

  26. Getting StrategicSo what are you going to create?

  27. Microsoft Azure Services for IoT Orientation to IoT Organization Characteristics Azure Services Buy Intelligent Systems Service (ISS) Solution • Building software is not a core competency • Focus on time-to-value • Custom protocols and security models • Custom analytics and data processing • Compose higher level services ISS Platform Services Buy Some, Build Some Build • Core competency in building software/services • Focus on flexibility & control, cost minimization • Strategic investment in custom service platform Core/App Platform Services

  28. The Value of IoT on Azure • Mix and match services to build your solution • Use components on any platform • Storm on Linux in a VM cluster • HDInsight service on Azure • Web Sites & Power BI

  29. Get started Azure Event Hub – IOT at Scale w/ Brent Stineman Monday@1:00pm, Guava Visit azure.microsoft.com Massively Parallel Azure Systems w/ Jason Young Tuesday@10:30am, Aralia HDInsight and Internet of Things w/ Tony Guidici Tuesday, 2:30pm, Tamarind

More Related