SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

ICS 421 Spring 2010 Parallel & Distributed Databases

alec avatar
alec
12 diapositives · 348 Views · Jul 26, 2014

ICS 421 Spring 2010 Parallel & Distributed Databases. Asst. Prof. Lipyeow Lim Information & Computer Science Department University of Hawaii at Manoa. Why Parallel Data Access ?. SELECT * FROM mydata. 1 Terabyte (= 1024 GB). 10 MB/s. 1000 x parallel 10 MB/s. 1.2 days to scan.

ICS 421 Spring 2010 Parallel & Distributed Databases - slide 1
01 / 12ICS 421 Spring 2010 Parallel & Distributed Databases
1 / 17

Télécharger la présentation

ICS 421 Spring 2010 Parallel & Distributed Databases
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. ICS 421 Spring 2010Parallel & Distributed Databases Asst. Prof. Lipyeow Lim Information & Computer Science Department University of Hawaii at Manoa Lipyeow Lim -- University of Hawaii at Manoa

  2. Why Parallel Data Access ? SELECT * FROM mydata 1 Terabyte (= 1024 GB) 10 MB/s 1000 x parallel 10 MB/s 1.2 days to scan 1.5 mins to scan! Lipyeow Lim -- University of Hawaii at Manoa

  3. Multi-Petabyte Databases How large is a petabyte? Lipyeow Lim -- University of Hawaii at Manoa

  4. Parallel DBMS • eBay’s main Teradata data warehouse (DW): • > 2 petabytes of user data • 10s of 1000s of users • Millions of queries per day • 72 nodes • >140 GB/sec of I/O, or 2 GB/node/sec • eBay’s Greenplum DW • 6 1/2 petabytes of user data • 96 nodes • 200 MB/node/sec of I/O • Walmart – 2.5 petabytes • Bank of America – 1.5 petabytes • Some parallel DBMSs besides the usual Oracle-IBM-MS trio: • Teradata • Netezza • Vertica • DATAllegro • Greenplum • Aster Data • Infobright • Kognitio, Kickfire, Dataupia, ParAccel, Exasol, ... Lipyeow Lim -- University of Hawaii at Manoa

  5. Parallelism Pipeline parallelism Partition parallelism many machines doing the same thing to different pieces of data. • many machines each doing one step in a multi-step process. Parallelism is natural to DBMS processing Processing Query 1 Node 1 Node 2 Node 3 Node 1 Q1:step1 Q1:step2 Q1:step3 Q2:step1 Q2:step2 Q2:step3 Node 2 Q3:step1 Q3:step2 Q3:step3 Q4:step1 Q4:step2 Node 3 Q5:step1 Lipyeow Lim -- University of Hawaii at Manoa

  6. Parallelism Terminology Ideal • Speed-up • Same job + more resources = less time • Scale-up • Bigger job + more resouces = same time • Transaction scale-up • More clients + more resources = same time Xact/sec. (throughput) degree of parallelism Ideal sec./Xact (response time) degree of parallelism Lipyeow Lim -- University of Hawaii at Manoa

  7. Parallel Architecture : Share What? Shared Memory (SMP) Shared Nothing (network) Shared Disk CLIENTS CLIENTS CLIENTS Processors Memory Hard to program Cheap to build Easy to scaleup Easy to program Expensive to build Difficult to scaleup Sequent, SGI, Sun VMScluster, Sysplex Tandem, Teradata, SP2 Lipyeow Lim -- University of Hawaii at Manoa

  8. Different Types of DBMS Parallelism • Intra-operator parallelism • get all machines working to compute a given operation (scan, sort, join) • Inter-operator parallelism • each operator may run concurrently on a different site (exploits pipelining) • Inter-query parallelism • different queries run on different sites • We’ll focus on intra-query parallelism Lipyeow Lim -- University of Hawaii at Manoa

  9. Parallel vs Distributed DBMS • A parallel database system • Improve performance via parallelization of various operations such as loading data, building indexes, evaluating queries • A distributed database system • Data is physically stored across several (geographical) sites • Each site is managed by an independent DBMS • Distribution governed by factors like local ownership & increased availability • The boundaries of these traditional definitions are blurring. Lipyeow Lim -- University of Hawaii at Manoa

  10. Types of Distributed DBMS • Homogeneous: Every site runs same type of DBMS. • Parallel DBMSs are usually homogeneous • Heterogeneous: Different sites run different DBMSs (different RDBMSs or even non-relational DBMSs). Gateway DBMS 1 DBMS 2 DBMS 3 Lipyeow Lim -- University of Hawaii at Manoa

  11. Data Partitioning & Fragmentation TID • Parallel DB • Data partitioning • Distributed DB • Fragmentation • Same basic problem : How do we break up the data (tables) and spread them amongst the “nodes” • Horizontal vs Vertical • Range vs Hash • Replication • DB user’s view should be one single table. t1 t2 t3 t4 Lipyeow Lim -- University of Hawaii at Manoa

  12. Automatic Data Partitioning Partitioning a table: RangeHashRound Robin • Shared disk and memory less sensitive to partitioning, • Shared nothing benefits from "good" partitioning A...E F...J F...J T...Z T...Z K...N K...N O...S T...Z F...J K...N O...S A...E O...S A...E Good for equijoins, range queries group-by Good for equijoins Good to spread load Lipyeow Lim -- University of Hawaii at Manoa