1 / 17

Mastering ETL: Extract, Transform, Load for Data Analytics

Learn the crucial process of Extract, Transform, Load (ETL) for effective data analytics. Explore data consistency challenges, organizational issues, and data quality assurance strategies. Gain insights into managing real-time and historical transactional data.

ziva
Télécharger la présentation

Mastering ETL: Extract, Transform, Load for Data Analytics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MIS2502:Data AnalyticsExtract, Transform, Load David SchuffDavid.Schuff@temple.eduhttp://community.mis.temple.edu/dschuff

  2. Where we are… Now we’re here… Data entry Transactional Database Data extraction Analytical Data Store Data analysis Stores real-time transactional data Stores historical transactional and summary data

  3. Getting the information into the data mart Now let’s address this part…

  4. Extract, Transform, Load (ETL)

  5. The Actual Process Extract Transform Load Transactional Database 1 Query Data conversion Query Data Mart Transactional Database 2 Data conversion Query Query Dimensional database Relational database

  6. ETL’s Not That Easy!

  7. Data Consistency: The Problem with Legacy Systems • An IT infrastructure evolves over time • Systems are created and acquired by different people using different specifications • This can happen through: • Changes in management • Mergers & Acquisitions • Externally mandated standards • Generally poor planning

  8. This leads to many issues What are the problems with each of these ?

  9. What’s the big deal?

  10. Now think about this scenario Hotel Reservation Database Café Database

  11. Solution: “Single view” of data • The entire organization understands a unit of data in the same way • It’s both a business goal and a technology goal but it’s really more this… ...than this

  12. Closer look at the Guest/Customer Guests Guest_number Guest_firstname Guest_lastname Guest_address Guest_city Guest_zipcode Guest_email Customer Customer_number Customer_name Customer_address Customer_city Customer_zipcode vs.

  13. Organizational issues

  14. Data Quality The degree to which the data reflects the actual environment

  15. Finding the right data Adapted from http://www2.ed.gov/about/offices/list/os/technology/plan/2004/site/docs_and_pdf/Data_Quality_Audits_from_ESP_Solutions_Group.pdf

  16. Ensuring accuracy Adapted from http://www2.ed.gov/about/offices/list/os/technology/plan/2004/site/docs_and_pdf/Data_Quality_Audits_from_ESP_Solutions_Group.pdf

  17. Reliability of the collection process Adapted from http://www2.ed.gov/about/offices/list/os/technology/plan/2004/site/docs_and_pdf/Data_Quality_Audits_from_ESP_Solutions_Group.pdf

More Related