1 / 25

Achieving Adaptivity for OLAP-XML Federations

Achieving Adaptivity for OLAP-XML Federations. Torben Bach Pedersen Aalborg University Joint work with Dennis Pedersen, TARGIT. Overview. Background: OLAP-XML federations New challenges XML data changes Slow or unreliable XML sources Schema changes in data sources Other challenges

kyne
Télécharger la présentation

Achieving Adaptivity for OLAP-XML Federations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Achieving Adaptivity for OLAP-XML Federations Torben Bach Pedersen Aalborg University Joint work with Dennis Pedersen, TARGIT

  2. Overview • Background: OLAP-XML federations • New challenges • XML data changes • Slow or unreliable XML sources • Schema changes in data sources • Other challenges • Integration in TARGIT architecture • Other applications of the techniques • Conclusion and future work • Related work Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  3. Data Warehousing & OLAP Multidimensional analysis: TARGIT Analysis Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  4. OLAP • Good for complex ad hoc queries • Simple: natural, graphical queries • Fast: pre-aggregation • A number of problems with physical integration • Short-term and varying data needs • Population, product info, ... • Dynamical data • Stock quotes, competitor pricing, ... • Data with limited access • Competitor product info, public databases, ... Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  5. Client OLAP-server Cube OLAP-XML Federations Traditional OLAP architecture: Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  6. OLAP-XML Federations • Logical integration of XML data • External dimensions • External measures • Data combined at query time Client Federation XML Cube Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  7. Client Federation XML Cube Cube OLAP-XML Federations • Logical integration of XML data • External dimensions • External measures • Data combined at query time • Transparent for users • Flexible: many XML sources • Quick: running in a few mins • Data is always fresh • Performance often comparable to physical integration Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  8. Client Federation XML Cube Cube XPath Queries for Fetching XML <Books> <Book> <Title>1984</Title> <Author>Orwell</Author> </Book> <Book> <Title>Of Mice and Men</Title> <Author>Steinbeck</Author> </Book> </Books> /Books/Book[Author=”Steinbeck”]/Title XPath Dimension value Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  9. Old And New TARGIT Architecture Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  10. New Challenges • Our previous work focused on basic aspects • Flexibility • General performance • Implementation • New: what can go wrong? – need for adaptivity • XML data changes • XML sources slow or unreliable • Schema changes (XML, OLAP, federation) • We often have no control over the XML sources • A solution has broad interest: views over XML sources Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  11. XML Data Changes • Basic federation • XML data is integrated at query time => XML data changes handled automatically • However, XML data is cached for performance • Cache timeout value ensures fresh data (set manually or automatically) • 0 cache timeout => always fetch from source • Only few current XML databases inform about changes • Xyleme allows users to subscribe to changes • Only delta should be transferred Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  12. ICE: Information and Content Exchange • Protocol proposed by W3C for automatically informing about and requesting changes • Supported by major vendors • Push: subscribe to changes and keep cache up-to-date • Pull: request changes from source at query time Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  13. Source Server Client Slow and Unreliable XML Sources • Overload, maintenance, HW breakdown, attacks • Often we no influence on this • Incremental presentation for user • What if source is too slow or no reply at all? • Inform user that the system is not working…? • Specification of alternative sources • Several queries per external dimension/measure • Increased fault tolerance, also better performance Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  14. Fed Fed 3 1 ? 1 1 Fed 2 Fed 2 Fed 2 3 3 Slow and Unreliable XML Sources • Start several queries and use the fastest • Always uses the fastest, but heavy load on sources • Use first response time as indicator for total time • Start one query at a time • Minimal load on sources, but slower Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  15. Slow and Unreliable XML Sources Alternative sources of lower quality: better than no data? Alternatives • Expired cache data • Google, Xyleme, The WayBack Machine • Backup-disk, tape • Etc. Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  16. Slow and Unreliable XML Sources In practice? Sources with equal priority chosen at random Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  17. Result: Algorithm for Fetching XML Data Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  18. Experiments • 1st experiment: fetching a 137 KB dimension • Start 8 queries, when first 3 respond, (cancel) last 5, when fastest query finish, (cancel) remaining 2 • Fast reply = good indication of overall speed • 2nd experiment: search local cache, then Google cache Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  19. Schema Changes In XML Sources How to synchronize XML views after schema change? (solution described in separate paper) Bibliography Bibliography Book Publisher Publisher Author AName Price Book Author PName Title PName Price Title AName /Bibliography/Author[AName=”Orwell”]/Book/Title Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  20. Additional Challenges • Changes to federation schema • Cache may be invalidated • Discard affected cache results (unproblematic) • OLAP data changes • Cache may be invalidated • Less frequent than XML data changes => cache will often have expired anyway • OLAP schema changes • Federated schema may be invalidated • Rare and easy to detect (and correct) Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  21. Integrating Techniques - Architecture Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  22. Integrating Techniques – Query Processing • Query Evaluator splits query into XML+OLAP parts and determines query plan based on cost • Execution Engine coordinates and executes plan • Cache Manager maintains cache, e.g., through ICE • XML Component interface fetches XML data, chooses between available XML sources (Algorithm 1) • View Synchronizer handles schema changes • Metadata Manager manages info about external dimensions and measures + XML component characteristics Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  23. Other Applications All XPath-based views on XML data Links to parts of XML documents • Web pages • Documents (DocBook) • Software applications and many more… Automatic recreation of broken links Increased fault tolerance and performance using alternative sources ? Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  24. Conclusion and Future Work • Operational problems in OLAP-XML federations • XML data changes • Slow and unreliable XML sources • Using several sources (Algorithm 1) • Experiment with Algorithm1 • Techniques integrated into federation architecture • Schema evolution and other challenges • Future work • TARGIT implementation and testing • Using techniques in other applications Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

  25. Related Work • Data changes in XML/semistructured documents • Xyleme + Zhuge • Schema changes in scientific documents • Not XML • Adaptive/dynamic query optimization • Telegraph project • We use once per source, rather than per tuple • Does not consider one or more of: OLAP+XML concepts, schema changes, slow and unreliable sources • Own previous OLAP-XML work is not adaptive Torben Bach Pedersen · DOLAP 2003 · 12-07-2014

More Related