1 / 11

Beaconstac Analytics

Beaconstac is a proximity marketing and analytics platform for beacons<br>Several beacon specific events are defined to aid proximity marketing<br>The events include Camp on event, beacon exit event, region enter, region exit etc.Beaconstac analytics platform makes it easy for managers/marketers/developers to analyze event data.

DeZyre
Télécharger la présentation

Beaconstac Analytics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Big Data and Internet of things(IOT)

  2. Project Morpheus (Beaconstac Analytics) May 2015 Garima Batra Core Platform Engineer | MobStac

  3. A quick intro about Beaconstac 1 • Beaconstac is a proximity marketing and analytics platform for beacons • Several beacon specific events are defined to aid proximity marketing • The events include Camp on event, beacon exit event, region enter, region exit etc. • Beaconstac analytics platform makes it easy for managers/marketers/developers to analyze event data • Components include Beaconstac iOS/Android sdk, beaconstac portal

  4. Why Hadoop? 1 • Collect event logs generated from Beaconstac SDK usage • Needed a system to answer queries like • Heat map of beacons by the number of visits received in a specified time interval. • Heat map of beacons by the amount of time spent in a specified time interval. • Average time spent by users near different beacons • Last seen per user • Last seen per beacon • Analyzing data with custom attributes filters • Traversed path in an area by individual users

  5. Leveraging Amazon's EMR for Beaconstac Analytics 1 • Amazon's Streaming API for writing mapper and reducer functions in Python • Input - Copy programs to Amazon S3 • Output – Copy the processed/output data to S3 • Initial tests were run using Amazon's EMR console. Here you can define the following - • Cluster configuration – Name, Termination protection, Logging, logs location on S3 etc. • Software configuration – Hadoop AMI version, applications to be installed on startup etc. • Hardware configuration – Types of nodes – master, Core and Task • Security keys, allowed users • Bootstrap actions – Configure Hadoop, Custom actions etc. • Steps – Streaming program, Hive program, Pig program

  6. Integrating EMR in production 1

  7. Batch processing for Morpheus 1 AWS Data pipeline

  8. Deep dive into EMR startup and job submission 1

  9. How Does AWS Data Pipeline Work? 1 • Pipeline definition - specifies the business logic of your data management • AWS Data pipeline web service - interprets the pipeline definition and assigns tasks to workers to move and transform data. • Task runner - polls the AWS Data Pipeline web service for tasks and then performs those tasks.

  10. Morpheus version of Data pipeline 1 Copy the output to Elastic Search Copy logs from Kafka to S3 Run EMR jobs • Runs once every day • Inserts output in Elastic search • Runs every hour • Requires a Kafka consumer script • Runs once every day • Processes each job and produces output • Each job comprises of mapper and reducer scripts

  11. Settings file in each job 1 1 Questions?? Source: Lorem Ipsum

More Related