1 / 81

Big Data Hands-On Labs:

Big Data Hands-On Labs:. Or d ownload : Big Data Lite Virtual Machine. Oracle Big Data Appliance for Customers and Partners. Jean-Pierre Dijcks Oracle Big Data Product Management Paul Kent SAS VP Big Data. Oracle Big Data Appliance for Customers and Partners. 1.

Télécharger la présentation

Big Data Hands-On Labs:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Big Data Hands-On Labs: Or download: Big Data Lite Virtual Machine

  2. Oracle Big Data Appliance for Customers and Partners Jean-Pierre Dijcks Oracle Big Data Product Management Paul Kent SAS VP Big Data

  3. Oracle Big Data Appliance for Customers and Partners 1 Big Data Appliance Recap Why You Should Consider Big Data ApplianceDriving Business Value with SAS on Big Data Appliance Q&A 2 3 4

  4. Oracle Big Data Management System Oracle Big Data SQL Oracle Database Oracle IndustryModels Oracle Advanced Analytics Oracle Spatial & Graph Cloudera Hadoop Oracle NoSQL Database Oracle R Advanced Analytics for Hadoop Oracle R Distribution Oracle Database Oracle Advanced Security Oracle Advanced Analytics Oracle Spatial & Graph Oracle Big DataConnectors Oracle DataIntegrator Big Data Appliance OracleExadata SOURCES

  5. Recap: Big Data Appliance Overview Big Data Appliance X4-2 Sun Oracle X4-2L Servers with per server: 2 * 8 Core Intel Xeon E5 Processors 64 GB Memory 48TB Disk space Integrated Software: Oracle Linux, Oracle Java VM Oracle Big Data SQL* Cloudera Distribution of Apache Hadoop – EDH Edition Cloudera Manager Oracle R Distribution Oracle NoSQL Database * Oracle Big Data SQL is separately licensed

  6. Recap: Standard and Modular • Starter Rack is a fully cabled and configured for growth with 6 servers • In-Rack Expansion delivers 6 server modular expansion block • Full Rack delivers optimal blend of capacity and expansion options • Grow by adding rack – up to 18 racks without additional switches

  7. Recap: Harness Rapid Evolution • BDA 4.0 – Sept 2014 • Big Data SQL • Node Migration • BDA 2.x – April 2013 • Starter Rack • In-Rack Expansion • EM Integration • BDA 3.x – April 2014 • CDH 5.0 (MR2 & YARN) • AAA Security • Encryption • BDA 1.0 – Jan 2012 • Initial BDA • Mammoth Install

  8. Core Design Principles for Big Data Appliance Operational Simplicity Simplify Access to ALL Data

  9. Core Design Principles for Big Data Appliance Operational Simplicity Simplify Access to ALL Data • Oracle Big Data SQL • Oracle SQL on ALL your data • All Native Oracle SQL Operators • Smart Scan for Optimized Performance • Oracle Security • Govern all Data through a Single Set of Security Policies

  10. Oracle Big Data SQL – A New Architecture • Powerful, high-performance SQL on Hadoop • Full Oracle SQL capabilities on Hadoop • SQL query processing local to Hadoop nodes • Simple data integration of Hadoop and Oracle Database • Single SQL point-of-entry to access all data • Scalable joins between Hadoop and RDBMS data • Optimized hardware • Balanced Configurations • No bottlenecks Oracle Confidential – Internal/Restricted/Highly Restricted

  11. Big Data SQL SELECT w.sess_id, c.name FROM web_logs w, customers c WHERE w.source_country = ‘Brazil’ AND w.cust_id = c.customer_id; Relevant SQL runs on BDA nodes Big Data SQL 10’s of Gigabytes of Data WEB_LOGS CUSTOMERS Only columns and rows needed to answer query are returned Hadoop Cluster Oracle Database

  12. Big Data SQL SELECT w.sess_id, c.name FROM web_logs w, customers c WHERE w.source_country = ‘Brazil’ AND w.cust_id = c.customer_id; • SQL Push Down in Big Data SQL • Hadoop Scans on Unstructured Data • WHERE Clause Evaluation • Column Projection • Bloom Filters for Better Join Performance • JSON Parsing, Data Mining Model Evaluation Relevant SQL runs on BDA nodes Big Data SQL 10’s of Gigabytes of Data WEB_LOGS CUSTOMERS Only columns and rows needed to answer query are returned Hadoop Cluster Oracle Database

  13. Oracle Communications Data Model Reference Architecture Oracle Comms Apps (BSS/OSS) ETL/ELT Adapters Customer Experience Big Data Platform(Hadoop/NoSQL) Oracle CommsNtwk Products (Tekelec & Acme) Real-Time Adapters Operations Other Oracle Apps (CRM, ERP, etc.) Relational Data Warehouse (OCDM) ThirdParty Monetization Third Party Sources Data Management Analytic Apps Adapters DataSources Feedback Loop To Other Apps

  14. Core Design Principles for Big Data Appliance Operational Simplicity Simplify Access to ALL Data

  15. Core Design Principles for Big Data Appliance Operational Simplicity Simplify Access to ALL Data • No Bottlenecks • Full Stack Install and Upgrades • Simplified Management • Cluster Growth • Critical Node Migration • Always Highly Available • Always Secure • Very Competitive Price Point

  16. Successful Big Data Systems Grow From Cluster Install with HA to Large Clusters to Dealing with Operational Issues Day 1 • 12 node BDA for Production • Hadoop HA and Security Set-up • Ready to Load Data Full install with a single command: ./mammoth –i rck_1 RCK_1

  17. Successful Big Data Systems Grow From Cluster Install with HA to Large Clusters to Dealing with Operational Issues Day 1 RCK_1 Example Service: Hadoop Name Nodes N N

  18. Successful Big Data Systems Grow From Cluster Install with HA to Large Clusters to Dealing with Operational Issues Day 90 Add 12 New Nodes across two Racks Cluster expansion with a single command: mammoth –e newhost1,…,newhostn RCK_2 RCK_1 N N

  19. Successful Big Data Systems Grow From Cluster Install with HA to Large Clusters to Dealing with Operational Issues Cluster Expansion with a single command: mammoth –e newhost1,…,newhostn RCK_2 RCK_1 This expansion automatically optimizes HA setup across multiple racks N Because of uniform nodes and IB networking,no data is moved N

  20. Successful Big Data Systems Grow From Cluster Install with HA to Large Clusters to Dealing with Operational Issues Day n Critical Node Failure => Primary Name Node RCK_2 RCK_1 N N

  21. Successful Big Data Systems Grow From Cluster Install with HA to Large Clusters to Dealing with Operational Issues RCK_2 RCK_1 N N Automatic Failover to other NameNode Automatic Service Request to Oracle for HW Failure

  22. Successful Big Data Systems Grow From Cluster Install with HA to Large Clusters to Dealing with Operational Issues RCK_2 RCK_1 N N Restore HA with a Single command bdacliadmin_cluster migrate N1 Reinstate the Repaired Node with a Single Command: bdacliadmin_clusterreprovision N1

  23. Mike Olson, Cloudera founder, Chief Strategy Officer, and Chairman of the Board Core Design Principles for Big Data Appliance Operational Simplicity 30% Quicker to Deploy “Oracle Big Data Appliance is an excellent choice for customers looking to work with the full suite of Cloudera’s leading Hadoop-based technology. It’s more cost-effective and quicker to deploy than a DIY cluster.” 21% Cheaper to Buy

  24. Big Data Initiative @ Oracle Global Support Services Real-time access to better data means better insights, which means better decisions and better business results Integrate data associated with customer telemetry, configurations, service history, diagnostics, knowledge & support information Anticipate Detect Predict Automate Delight

  25. Core Design Principles Enable Success Operational Simplicity Simplify Access to ALL Data

  26. There is one more thing… • Business Value = Applications

  27. Big Data Appliance powers instant Business Value Customer Experience Management CommunicationsData Model Cyber SecuritySolutions

  28. Introducing • Paul Kent - SAS

  29. Big Data and Big Analytics – So Much more Gunpowder! Paul Kent VP BigData, SAS Research and Development

  30. 1. Change 2. Safari Pics

  31. [CON8279] Oracle Big Data Appliance: Deep Dive and Roadmap for Customers and Partners Oracle Big Data Appliance is the premier Hadoop appliance in the market. This session describes the roadmap for customers in the areas of high-performance SQL on Hadoop and securing big data, plus overall performance improvements for Hadoop. A special focus in the session is the roadmap and benefits Oracle Big Data Appliance brings to Oracle partners. To illustrate the benefits of running on a standardized and optimized Hadoop platform, SAS presents the findings of its tests of SAS In-Memory Analytics on Oracle Big Data Appliance.

  32. SAS & Oracle Partnership Family Stories Hadoop Oracle Engineered Systems Family SAS Software Family Deployment Patterns Agenda

  33. Reflection on a stronger partnership than ever • Both leaders in Big Data – • Jointly solving the most difficult and demanding Big Data Problems • Providing simplicity and agility to create flexible configurations • Extensive engineering collaboration • Can we answer: • How Does it Work? • How Does it Perform? 2014

  34. the tamoxifen dilemma SOURCE: http://commons.wikimedia.org/wiki/File:Tamoxifen-3D-vdW.png

  35. SAS & Oracle Partnership Family Stories Hadoop Oracle Engineered Systems Family SAS Software Family Deployment Patterns Agenda

  36. Elephant :: 3 Good Ideas !! Never forgets Is a good (hard) worker Is a Social Animal (teamwork)

  37. MPP (Massively Parallel) hardware running database-like software “data” is stored in parts, across multiple worker nodes “work” operates in parallel ,on the different parts of the table Hadoop – Simplified View Controller Worker Nodes

  38. Idea #1 - HDFS. Never forgets!

  39. Idea #1 - HDFS. Never forgets!

  40. Idea #1 - HDFS. Never forgets! X X

  41. Redundancy Wins!

  42. Idea #2 – MapReduce – Send the work to the Data • We Want the Youngest Person in the Room • Each Row in the audience is a data node • I’ll be the coordinator • From outside to center, accumulate MIN • Sweep from back to front. • Youngest Advances

  43. SAS & Oracle Partnership Family Stories Hadoop Oracle Engineered Systems Family SAS Software Family Deployment Patterns Agenda

  44. Recap: Standard and Modular • Starter Rack is a fully cabled and configured for growth with 6 servers • In-Rack Expansion delivers 6 server modular expansion block • Full Rack delivers optimal blend of capacity and expansion options • Grow by adding rack – up to 18 racks without additional switches

  45. Oracle Big Data SQL – A New Architecture • Powerful, high-performance SQL on Hadoop • Full Oracle SQL capabilities on Hadoop • SQL query processing local to Hadoop nodes • Simple data integration of Hadoop and Oracle Database • Single SQL point-of-entry to access all data • Scalable joins between Hadoop and RDBMS data • Optimized hardware • Balanced Configurations • No bottlenecks Oracle Confidential – Internal/Restricted/Highly Restricted

  46. Diversity. It’s a good thing! Impala Nyala

  47. SAS & Oracle Partnership Family Stories Hadoop Oracle Engineered Systems Family SAS Software Family Deployment Patterns Agenda

  48. 4 Important Things #1 Join the Family

  49. SAS ACCESS to Hadoop HADOOP SAS SERVER Hive QL #2 Be Familiar

More Related