Database Management Systems

1. Database Management Systems Chapter 10 Distributed Databases

2. Definition Advantages / Uses Problems / Complications Client-Server / SQL Server Microsoft Access Distributed Databases SELECT Sales FROM Britain.Sales UNION SELECT Sales FROM France.Sales UNION SELECT Sales FROM Italy.Sales Germany Britain France Italy

3. Multiple independent databases Each DBMS is a complete DBMS (engine, queries, locking, transactions, etc.) Usually on different machines. Usually in different locations. Connected by a network. Might be different environments Hardware Operating System DBMS Software Distributed Database Definition Database Apollo Database Zeus England France Database Athena United States

4. C.J. Date Rule 0: Transparency: the user should not know or care that the database is distributed. Local autonomy. No reliance on a central site. Continuous operation. Location independence. Fragmentation independence (physical storage). Replication independence. Distributed query processing. Distributed transaction management. Hardware independence. Operating system independence. Network independence. DBMS independence. Distributed Database Rules

5. Each database can continue to run even if portion fails. Data and hardware can be moved without affecting operations or users. Expanding operations. Performance issues. System expansion and upgrades. Add new section without affecting others. Upgrade hardware, network and DBMS. Distributed Features

6. Business operations are often distributed Work and data are segmented by department. Work and data are segmented by geographical location. Improved performance Most updates and queries are performed locally. Maintain local control and responsibility over data. Can still combine data across the system. Scalability and expansion Add on, not replacement. Advantages and Applications local transactions future expansion

7. Creating a Distributed Database • Design administration plan. • Choose hardware and DBMS vendor, and network. • Set up network and DBMS connections. • Choose locations for data. • Choose replication strategy. • Create backup plan and strategy. • Create local views and synonyms. • Perform stress test: loads and failures.

8. Networks are slow Drives: 20 - 60 MB per sec. LANs: 1-10 MB per sec (10-100 mbps). WANs: 0.01 - 5 MB per sec. Faster is possible but expensive! SANs: 10-100 MB per sec. Goal is to minimize transmissions. Each system must be capable of evaluating queries--preferably SQL. Results depend heavily on how the system joins tables. Distributed Query Processing WAN 0.1 - 5 MB 10-100 MB LAN 10 - 20 MB Disk drive

9. Example NY: Customers: 1 M rows LA: Production: 10 M rows Chicago: Sales: 20 M rows Query: List customers who bought blue products on March 1 Bad idea #1 Transfer all rows to Chicago Then JOIN and select. Better idea #2 (probably) Transfer blue products from LA to Chicago Better idea #3 Get sale items on March 1 Get blue products from LA Send C# to NY Distributed Query Processing NY Customers(C#, …) 1,000,000 C# list from desired P# Chicago Matching Customer data Sales(S#, C#, Sdate) 20,000,000 SaleItem(S#, P#,…) 50,000,000 P# sold on March 1 Blue P# sold on March 1 LA Products(P#, Color…) 10,000,000

10. Goals Minimize transmissions Improve performance Support heavy multiuser access. Problems Updating copies Bulk transmissions Site unavailable Concurrency Easier for two people to change the same data at the same time. Decision support systems. Data warehouse. Data Replication Market research & data corrections. Britain Britain: Customers & Sales France: Customers & Sales Spain: Customers & Sales Periodic updates Spain Britain: Customers & Sales France: Customers & Sales Spain: Customers & Sales Update data.

11. Each DBMS must maintain lock facility. To update, each DBMS must utilize and recognize other lock mechanisms and return codes. Each DBMS must have a deadlock resolution protocol that recognizes the distributed databases. Random wait. Optimistic updates. Two-phase commit. Concurrency and Locks DBMS #1 Accounts Jones 8898 Transaction A Locked Waiting Transaction B Waiting Locked DBMS #2 Accounts Jones 3561

12. Two (or more) separate lock managers. DBMS initiating update serves as the coordinator. Two phases Coordinator sends message and data to all machines to “get ready.” Local machines save data in logs, verify update status and return message. If all locals report OK, then coordinator writes log and instructs others to proceed. If any fail, it sends Rollback message. Transactions & Two-Phase Commit Database 1 Initiate Transaction 1. Prepare to commit. All agree? 2. Commit Database 2 Lock tables. Save log. Database 3 Update all tables.

13. Distributed Transaction Managers Transaction Manager Transaction Manager Resource Manager Resource Manager DBMS DBMS Transaction Manager Transaction Processing Monitor Resource Manager DBMS The distributed transaction coordinator/transaction processing monitor handles the transaction decisions and coordinates across the participating systems.

14. Distributed Design Questions

15. Database Links Full database names. CONNECT command. Linking through synonyms. CREATE SYNONYM … Central control over permissions. Linking through Views/queries. CREATE VIEW AS … Can assign local permissions. Linking through stored procedures. DELETE … Strong control over actions. Distributed Databases In Oracle Schema.Table@Location Scott.Emp@hq.acme.com Server database Synonym: Employee Procedure: DELETE FROM Employee WHERE ... View User can only run procedure. No other access. user permissions

16. Client-Server Server Server Shared Database Front-end User Interface Clients Clients

17. Not a distributed database. Data file stored on server. Server is passive, appears as giant disk drive to PC. PC processes all data. Retrieves all needed data across the network. Performance improvements. Indexes are crucial. Store some data on each PC (replication). Store applications on PC (graphics & forms). Convert to SQL-Server LAN File Server File Server DBMS data file Application Shared Data All data from all tables are read by PC, which performs JOIN and WHERE test. If available, reads index first. SELECT Name, SaleDate FROM Customer INNER JOIN Sales ON Customer.C# = Sales.C# WHERE SaleDate BETWEEN #1-Mar-97# AND #9-Mar-97#;

18. LAN File Server: Slow File Server MyFile.mdb Forms CustID Name … 115 Jenkins … 125 Juarez ... Order ... Application and query transferred. DBMS software transferred. One row at a time transferred, until all rows are examined. SELECT * FROM Customer WHERE City = “Sandy”

19. One machine machine is dominant (server) and handles data for many clients. Client machines handle front-end tasks and small data tables that are not shared. Client-Server Databases File Server DBMS SQL Server Shared Data Return matching data. Send SQL statement. SELECT . . . application

20. ADO and Direct Connections Server Computer Database Server The Database vendor provides its own data transport (e.g,. Oracle or SQL Server) installed on the server and the client. ADO provides a driver that connects your application to the transport services. ODBC can serve as the data transport if nothing else is available DBMS transport SELECT … Results Client Computer DBMS transport ADO Visual Basic application

21. Server Databases Client front-end Middle Locate databases Business rules Program code Three-Tier Client-Server Database Servers Databases. Transactions. Legacy applications. Database links. Business rules. Program code. Middleware Application. Front-end. User Interface. Client

22. Database Independence on the Client Original DBMS New DBMS ADO ADO Application

23. Database Independence with Queries Independent Application Query: works with any DBMS SELECT SaleID, SaleDate, CustomerID, CustomerName FROM SaleCustomer Saved Oracle Query SELECT SaleID, SaleDate, CustomerID, LastName || ‘, ‘ || FirstName AS CustomerName FROM Sale, Customer WHERE Sale.CustomerID=Customer.CustomerID Saved SQL Server Query SELECT SaleID, SaleDate, CustomerID, LastName + ‘, ‘ + FirstName AS CustomerName FROM Sale INNER JOIN Customer ON Sale.CustomerID = Customer.CustomerID

24. The Internet as Client-Server information Internet Router Router Server request Client Browser Web Server http://server.location/page HTML pages Forms Graphics

25. HTML Limited Clients <HTML> <HEAD> <TITLE>My main page</TITLE></HEAD> <BODY BACKGROUND=“graphics/back0.jpg”> <P>My text goes in paragraphs.</P> <P>Additional tags set <B>boldface</B> and <I>Italic</I>. <P>Tables are more complicated and use a set of tags for rows and columns.</P> <TABLE BORDER=1> <TR><TD>First cell</TD><TD>Second cell</TD></TR> <TR><TD>Next row</TD><TD>Second column</TD></TR> </TABLE> <P>There are form tags to create input forms for collecting data. But you need CGI program code to convert and use the input data.</P> </BODY> </HTML>

26. HTML Output

27. Web Server Database Fundamentals Request Server/Form.html 0 3 Client/Browser Database Page = Template + Result Data 2 1 2 3 CGI String DBMS HTML Form Result 1 Query Form Web Server Result Page 1 2 HTML form Query Template + Code Form.html Program code

28. Database Example: Client Side Request Server/Form.html 0 Server Initial form 1 Results 3 Call ASP page 2

29. Client-Server Data Transfer Order Form Order ID 1015 Jones, Martha Customer Order Date 12-Aug What if there are 10,000 customers? How much time to load the combo box? How do you refresh/reload the combo box? Alternatives?

30. Latency Server Generate form Receive form data Transmission delay Transmission delay time Form received Client User delay

31. XML: Transferring Data Order: OrderID, OrderDate, ShippingCost, Comment Item: ItemID, Description, Quantity, Cost Item: ItemID, Description, Quantity, Cost Item: ItemID, Description, Quantity, Cost Many XML files contain hierarchical data.

32. XML: Schema Definition xsd <?xml version="1.0" encoding="utf-8"?> <xs:schema id="OrderList" xmlns="" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata"> <xs:element name="OrderList" msdata:IsDataSet="true"> <xs:complexType> <xs:choice maxOccurs="unbounded"> <xs:element name="Order"> <xs:complexType> <xs:sequence> <xs:element name="OrderID" type="xs:string" minOccurs="0" /> <xs:element name="OrderDate" type="xs:date" minOccurs="0" /> <xs:element name="ShippingCost" type="xs:string" minOccurs="0" /> <xs:element name="Comment" type="xs:string" minOccurs="0" /> <xs:element name="Items" minOccurs="0" maxOccurs="unbounded"> <xs:complexType> <xs:sequence> <xs:element name="ItemID" nillable="true" minOccurs="0" maxOccurs="unbounded"> <xs:complexType> <xs:simpleContent msdata:ColumnName="ItemID_Text" msdata:Ordinal="0"> <xs:extension base="xs:string"> </xs:extension> </xs:simpleContent> </xs:complexType> </xs:element> <xs:element name="Description" nillable="true" minOccurs="0" maxOccurs="unbounded"> <xs:complexType> <xs:simpleContent msdata:ColumnName="Description_Text" msdata:Ordinal="0"> <xs:extension base="xs:string"> </xs:extension> </xs:simpleContent> </xs:complexType> </xs:element> Partial file, generated by .NET xsd.exe

33. <?xml version="1.0"?> <!DOCTYPE OrderList SYSTEM "orderlist.dtd"> <OrderList> <Order> <OrderID>1</OrderID> <OrderDate>3/6/2004</OrderDate> <ShippingCost>$33.54</ShippingCost> <Comment>Need immediately.</Comment> <Items> <ItemID>30</ItemID> <Description>Flea Collar-Dog-Medium</Description> <Quantity>208</Quantity> <Cost>$4.42</Cost> <ItemID>27</ItemID> <Description>Aquarium Filter &amp; Pump</Description> <Quantity>8</Quantity> <Cost>$24.65</Cost> </Items> </Order> </OrderList> XML Data Example XML: extensible markup language

34. XML Example in Explorer

35. Java and JDBC Connection con = DriverManager.getConnection( "jdbc.myDriver:myDBName", “myLogin”, “myPassword”); Statement smt = con.CreateStatement(); ResultSet rst = smt.executeQuery( “SELECT AnimalID, Name, Category, Breed FROM Animal”); while (rst.next()) { int iAnimal = rst.getInt(“AnimalID”); String sName = rst.getString(“Name”); String sCategory = rst.getString(“Category”); String sBreed = rst.getString(“Breed”); \\ Now do something with these four variables }