1 / 48

Overview and Direction of the IBM Tivoli System Automation Product Family

Overview and Direction of the IBM Tivoli System Automation Product Family. Creighton Hicks hicksc@us.ibm.com. Agenda. Business Case Tivoli System Automation Product Family Tivoli System Automation for z/OS Tivoli System Automation for Multiplatforms Base Pre-Canned Policy Templates

frances-prince
Télécharger la présentation

Overview and Direction of the IBM Tivoli System Automation Product Family

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Overview and Direction of theIBM Tivoli System Automation Product Family Creighton Hicks hicksc@us.ibm.com

  2. Agenda • Business Case • Tivoli System Automation Product Family • Tivoli System Automation for z/OS • Tivoli System Automation for Multiplatforms Base • Pre-Canned Policy Templates • Tivoli System Automation for Multiplatforms End-to-End • What’s next on the Scope

  3. Outage Causes Cost per hour for being Down in M$ Application Operator Failures 40% 40% Errors 40% 40% Brokerage 20% 20% Retail Sales Pay per view IDC 2001 Technology Failures 0 1 2 3 4 5 6 7 Standish Group Business Issue On demand challenges • Downtime unaffordable • Heterogeneous by nature • Different HW/SW platforms • Cross-cluster application dependencies • Complexity Customer pressures • No end-to-end automation • Cost and availability issues due to multiple automation and operations teams • Education requirements • Automation implementation and maintenance costs • Rapid change of I/T infrastructure • Loss of business • Lossof customers – the competition is just a mouse click away • Lossof credibility, brand image and stock value

  4. ’87 ’88 ’99 1990 ’91 ’92 ’93 ’94 ’1995 ’96 ’97 ’98 ’99 2000 ’01 ’02 ’03 ’04 2005 ’06 ’07 Towards a System Automation Family SA MP (End-to-End Autom.) Cross Cluster Automation TSA for Multiplatforms SA MP 1.2 SA MP 2.2 One Product SA MP (Base) HA for Distributed Systems SA for Linux 1.1 SA MP 2.1 SA z/OS Automation and HA ESCON Manager SA z/OS ISCF ACA/ISCF TSCF AOC GDPS AOC 1.4 AOC 1.2 SA z/OS 3.1 SA z/OS 2.2 AOC 1.1 SA z/OS 3.2 AOC 1.3 SA z/OS 2.3 SA z/OS 2.1 AF/REMOTE

  5. HA Scope HA Scope SAP DB2 Scripts,... HA Scope HAScope HA Scope J2EE Appl WAS DB2 WebSrv Scripts,... Scripts,... Landscape “HA View” – Today often Bottom Up Sample Bank CRM System Trading System

  6. HA Scope HA Scope SAP DB2 Dependency HA Scope HAScope HA Scope J2EE Appl WAS DB2 WebSrv Dependency Dependency Enterprise View – Top Down Sample Bank CRM CRM System Trading System Trading System

  7. Automation and High Availability withIBM Tivoli System Automation for z/OS

  8. Agent CF Agent Agent SA z/OS’s Architecture • Automation Manager • Knows sysplex wide configuraiton • Drives automation decisions • Implements an expert system tuned for automation concepts • Applies automation concepts with a high abstraction agains abstract resources: • Several group types • Several dependency types • Automation Agent • Starts, stops, and monitors resources Abstract Resource Model Automation Manager

  9. High Availability withIBM Tivoli System Automation for Multiplatforms(Base Component)

  10. Globalized Resources SA MP (base) - Mainframe-like High Availability for the Distributed World Automation Manager Exploits proven, leading pSeries & zSeries technology: • AIX’s cluster infrastructure RSCT (Reliable Scalable Cluster Technology) • provides functions like heartbeat, resource monitoring and control, … • z/OS’ Automation Manager • Expert system (hidden to the user) which drives automation decisions based on an automation policy Automation Manager Recovery RM Recovery RM Recovery RM RSCT RMC RMC RMC GblRes RM FileSys RM Config RM other RM GblRes RM FileSys RM Config RM other RM GblRes RM FileSys RM Config RM other RM .. .. .. HAGS HAGS HAGS HATS HATS HATS Node 1 Node 2 Node 3

  11. Decide Automation Policy Discovery correlation rules Resources P Relationships Resources & Relationships & Rules A Submit Action Find actions to reach Evaluate relationship Apply state goal correlation rules change Translate Action Translate event Request to a state Monitor Events Changes M E Monitor and Control Resources Tivoli System Automation – Basic Functionality

  12. Windows on PCs (xSeries and non-IBM PCs) Linux on PCs (xSeries and non-IBM PCs) Solaris on SUN-servers AIX and Linux on pSeries z/OS on zSeries Linux on zSeries Linuxon BladeCenter HP-UX on HP-servers Linux on iSeries SA MP (Base) Towards a Distributed World PC Systems “classical” UNIX Servers zSeries Currently under discussion Currently under discussion SA MP (base) SA MP (base) SA z/OS SA MP (base) SA MP (base) SA MP (base)

  13. Resource Group: DemoRG Floating Resource: WebIP WebIP WebIP DependsOn Floating Resource: WebServer WebServer WebServer charmhost1 charmhost2 Simple Example: How do I automate my Cluster? Step I: Define and start the cluster preprpnode charmhost1 charmhost2 mkrpdomain democluster charmhost1 charmhost2 startrpdomain democluster Step II: Define resources Step IIa: Define the WebServer resource mkrsrc IBM.Application Name="WebServer" StartCommand="/opt/IBMHTTPServer/bin/apachectl start" StopCommand="/opt/IBMHTTPServer/bin/apachectl stop" MonitorCommand="/root/build/FVT/Utils_386/ingcmon -USERID root -PATH /opt/IBMHTTPServer/bin/httpd" MonitorCommandPeriod=5 MonitorCommandTimeout=5 NodeNameList="{'charmhost1','charmhost2'}" UserName="root" RunCommandsSync=0 Step IIb: define the WebIP resource mkrsrc IBM.ServiceIP Name="WebIP" IPAddress="9.152.140.20" NetMask="255.255.255.0" NodeNameList="{'charmhost1','charmhost2', '}" Step III: Define automation policy Step IIIa: Create a resource group DemoRG with members WebServer and WebIP mkrg DemoRG addrgmbr -m T -g DemoRG IBM.Application:WebServer addrgmbr -m T -g DemoRG IBM.ServiceIP:WebIP Step IIIb: Define a relationship (as example) mkrel -p DependsOn –S IBM.ServiceIP:WebIP -G IBM.Application:WebServer

  14. Resource Types and Policy Elements “Everything is a resource” • “Simple” Resourcess • Serial fixed resource • Serial floating resource • Resource Group • Is a collection of resources which are treated as one logical instance • Is used to start, stop, and monitor • Group status is an aggregation of its members‘s status • Members can be Resources and Resource Groups • Equivalency • Relationships • For start/stop sequence: StartAfter, StopAfter • For dependend resources: DependsOn, DependsOnAny, and ForcedDownBy • For placement constraints: Collocated, AntiCollocated, Affinity, AntiAffinity, IsStartable Resource Group:RG_DB2 Floating Resource: ServiceIP ServiceIP ServiceIP DependsOn Floating Resource: DB2 DB2 DB2 DependsOn DependsOn Floating Resource: Mount Point MountPoint MountPoint Equivalency: Network NIC NIC NIC NIC Node 1 Node 2

  15. Resource Types Serial Fixed =unique resource in the cluster Serial Floating = one instance is started at one time. Fixed B Fixed D Serial Fixed Fixed A Fixed C Float E Fixed E1 Fixed E2 FixedE3 Serial Floating Float F Fixed F1 Fixed F2 Node 1 Node 2 Node 3

  16. Resource Group • Is a collection of resources which are treated as one logical instance • Is used to start, stop, and monitor • Start/stop of a group affects all its members • Group status is the aggregation of its members’ status • Resource Group Members can be • Resources • Resource Groups Start/Stop “chrg –o online <rg>” “chrg –o offline <rg>” Monitor “lsrg –g <rg>” RG_WebAppl Web Server Service IP File System

  17. Resource Groups - Nesting Start/Stop/Monitor “chrg –o online <rg>” “chrg –o offline <rg>” “lsrg –g <rg>” RG_AirlineReservation RG_WebAppl RG_DB RG_XX Web Server DB Instance RG_XX1 Appl C Appl E Service IP DB Data Appl D File System Service IP

  18. Relationships • Type I: Start sequence relationship • StartAfter • StopAfter • Type II: DependsOn relationships • DependsOn • DependsOnAny • ForcedDownBy • Type III: Location relationships • Collocated • AntiCollocated • Affinity • AntiAffinity • IsStartable

  19. StartAfter Start sequencing Policy Start Sequence Relationship Start Sequence Start Sequencing Appl A Appl A StartAfter Service IP Service IP

  20. Policy DependsOn Relationships • Main Idea: Appl A depends on the function of Appl B • If B is gone, A does not work • DependsOn • Start sequencing • Force down behavior • Stop sequencing • Implicit Collocation Start Sequence Force Down Stop Sequence on 1,2 Appl A Appl A Appl A DependsOn File System File System on 1,2 File System

  21. Policy Policy DependsOn Relationships • DependsOn • Start sequencing • Stop sequencing • Force down behavior • Implicit collocated • DependsOnAny • Start sequencing • Stop sequencing • Force down behavior on 1,2 Appl A DependsOn on 1,2 File System on 1,2 Appl A DependsOnAny on 3,4 DB2

  22. Location Relationships • Hard location constraints: • Collocated relationship • Resources must run on the same location • AntiCollocated relationship • Resource must run on different locations • Soft location constraints • Affinity relationship • Resources should run on the same location • AntiAffinity relationship • Resources should run on different locations

  23. Policy Collocated Relationship App App IP IP on 1,2,3,4 Node 1 Node 2 App Collocated IP App on 1,2,3,4 IP Node 3 Node 4

  24. Policy AntiCollocated Relationship Web Web on 1,2,3,4 Node 1 Node 2 Web AntiCollocated SAP SAP Web on 1,2,3,4 Node 3 Node 4

  25. Flexible Source/Target Definition • Possible to define relationships between... • resource groups, • resources, • equivalencies • ...in any combination RG_B RG_A RG_C Appl A Appl D Appl E Appl B Appl F

  26. Cross Node Relationships • Relationships can be defined between resources running on different systems in the cluster RG_Dist1 Fixed B Fixed A Float E Fixed E1 Fixed E2 FixedE3 RG_Dist2 Fixed C RG_Dist2 Fixed D FixedF Fixed G Node 1 Node 2 Node 3

  27. Pre-Canned Policy Templates

  28. Why do we need Pre-canned Policy Templates? • “ Well, TSA MP provides very powerful automation policy concepts with a very high abstraction level. So, why are pre-canned policy templates needed? ” • Good reasons for Pre-canned Policy Templates • Customer asks for solutions not products • Pre-canned policies simplify customer’s life and helps to save time • E.g.: No policy development, less testing, fast enablement,… • A customer often has no knowledge about application-specific HA requirements • Needed to define automation policy • A customer might not have knowledge about HA clustering

  29. Resource Group: DemoRG Floating Resource: WebIP WebIP WebIP DependsOn Floating Resource: WebServer WebServer WebServer charmhost1 charmhost2 Pre-Canned Policy Template: An Example Step I: Define and start the cluster preprpnode charmhost1 charmhost2 mkrpdomain democluster charmhost1 charmhost2 startrpdomain democluster Step II: Define resources Step IIa: Define the WebServer resource mkrsrc IBM.Application Name="WebServer" StartCommand="/opt/IBMHTTPServer/bin/apachectl start" StopCommand="/opt/IBMHTTPServer/bin/apachectl stop" MonitorCommand="/root/build/FVT/Utils_386/ingcmon -USERID root -PATH /opt/IBMHTTPServer/bin/httpd" MonitorCommandPeriod=5 MonitorCommandTimeout=5 NodeNameList="{'charmhost1','charmhost2'}" UserName="root" RunCommandsSync=0 Step IIb: define the WebIP resource mkrsrc IBM.ServiceIP Name="WebIP" IPAddress="9.152.140.20" NetMask="255.255.255.0" NodeNameList="{'charmhost1','charmhost2', '}" Step III: Define automation policy Step IIIa: Create a resource group DemoRG with members WebServer and WebIP mkrg DemoRG addrgmbr -m T -g DemoRG IBM.Application:WebServer addrgmbr -m T -g DemoRG IBM.ServiceIP:WebIP Step IIIb Define a relationship (as example) mkrel -p DependsOn –S IBM.ServiceIP:WebIP -G IBM.Application:WebServer Step II (New): Use Pre-Canned Policy Template Step IIa: Fill out Configuration File Step IIb: Trigger Policy Generator Pre-Canned Policy Template Configuration File # SAMBA example script_dir="/usr/sbin/rsct/sapolicies/samba" prefix="SA-samba-" daemonpath="/usr/lib/samba/classic" nodes="node-1 node-2" ip_1="192.168.2.22,255.255.255.0" nieq_1="eth0:node-1,eth0:node-2" data_work="/mnt Application-specific Policy Generator(script provided by IBM)

  30. Using Pre-canned Policy Templates – For Free Pre-canned policy templates • Are provided by IBM for a growing set of applications • Are “plug and play” • Are easy to adjust to customer specific requirements • Are tested by IBM • Contain all resource definitions and policy definitions • Provide a detail documentation • Pre-canned policy templates are for free !!!! • If no pre-canned policy is available for your specific application • IBM helps • Define your own policy

  31. Data Management DB2 8.x ESE DB2 8.x ESE DPF DB2 8.x HADR DB2 7.x WE, EE Oracle 9i Oracle 8i WebSphere WebSphere Application Server 6.0 WebSphere MQ Tivoli Products Tivoli Workload Scheduler CCMDB / TADDM Tivoli Storage Manager (TSM) TSM Client Tivoli Enterprise Console 3.8 SAP SAP Replicated Enqueue environment SAP Application Server Shared File Systems NFS Server NFS Client Samba Groupware Sendmail 8.11 Web Servers Apache Web Server IBM HTTP Server Currently under development: WebSphere Application Server 5.1 DP for mySAP DRBD SA MP End-to-End Component Tivoli Provisioning Manager Pre-canned Automation Policy Templates (Linux / AIX) http://www.ibm.com/software/tivoli/products/sys-auto-linux/enablement.html

  32. Resource Group: RG_DB2 Floating Resource: ServiceIP ServiceIP ServiceIP DependsOn Floating Resource: DB2 DB2 DB2 DependsOn Floating Resource: Mount Point MountPoint MountPoint Combining Pre-canned Policy Templates “Complete Application” mySAP + DB2 • TSA MP’s powerful policy elements allow easy integration (e.g. nested resource groups, or cross-node relationships) • Consider templates as a tool kid which can build up a “complete application” • Start, • stop, • and monitor mySAP Pre-Canned Policy Template DependsOn DB2 Pre-Canned Policy Template

  33. Pre-Canned Policy Template for Oracle

  34. Pre-canned Policy Template for DB2 • TSA MP’s pre-canned policy is DB2’s default HA solution on Linux • Shipped with every DB2 V8.2 CD (Linux) • Includes a free TSA MP License for a two-node cluster • Pre-canned policy template supports: • DB2 8.x ESE, • DB2 8.x ESE DPF (multi-partition) • DB2 8.x HADR • and DB2 7.x WE, EE • Documentation Available: • Highly Available DB2 Universal Database v8.x ESE/DPF using IBM Tivoli System Automation for Multiplatforms • Automating DB2 HADR Failover using IBM Tivoli System Automation for Multiplatforms

  35. Resource Group: RG_DB2 DB2 Pre-Canned Policy Template • Step I: Define and start cluster • preprpnode <node1> <node2> ... • mkrpdomain <domainname> <node1> <node2> ... • startrpdomain <domainname> • Step II: Use Pre-canned Policy Template • cd /opt/IBM/dbs/V8.1/ha/salinux • ./regdb2salin –a <DB2 Instance> -m <MountPoint> -i <IP Address> • (Exception: The DB2 pre-canned policy template works with options instead of a config file) Floating Resource: ServiceIP ServiceIP ServiceIP DependsOn Floating Resource: DB2 DB2 DB2 DependsOn DependsOn Floating Resource: Mount Point MountPoint MountPoint Equivalency: Network NIC NIC NIC NIC Node 1 Node 2

  36. XI_ABAP_LOP_ENQ ES MS db2_db2lop_0-rg XI_ABAP_LOP_siccps06_DV.. DB2 Instance ServiceIP GW XI_ABAP_LOP_siccps07_D00 ABAP AS siccps07 DependsOn DependsOn StartAfter CO DependsOn ServiceIP DB2_Mount SE SAP_SYS_ROUTER ServiceIP SAPROUTER SA-nfsserver-rg DependsOn SAPNFS DependsOn DependsOn SHARED DATA ServiceIP Pre-canned Policy Template for SAP: (ABAP, J2EE, XI, or ADIN) XI_ABAP_LOP_ENQREP StartAfter, IsStartable, AntiCollocated ERS Collocated IfNotOffline StartAfter StartAfter StartAfter StartAfter floating Resource (siccps06,siccps07) Resource Group fixed Resource

  37. Resource Group: tws#-rg Resource Group: tws#-rg Resource Group: tws#-rg IBM.Application:TWS-server-rs IBM.Application:TWS-server-rs IBM.Application:TWS-server-rs DependsOn DependsOn DependsOn IBM.Application:TWS-Appserv1-rs IBM.Application:TWS-Appserv1-rs IBM.Application:TWS-Appserv1-rs Pre-canned Policy Template for TWS Resource Group: TWS_mdm-rg IBM.Application:TWS-MDM-rs DependsOn Equivalency: TWS_server-equ IBM.Application: TWS-server1 IBM.Application: TWS-server1 IBM.Application:TWS-server2 . . . . . . DependsOnAny Resource Group: db2hadr_TWS-rg From DB2 HADR Pre-canned Policy

  38. Cross Cluster Automation withIBM Tivoli System Automation for Multiplatforms(End-to-End Automation Component)

  39. HA Scope HA Scope SAP DB2 Dependency HA Scope HAScope HA Scope J2EE Appl WAS DB2 WebSrv Dependency Dependency Enterprise View – Top Down Sample Bank CRM CRM System Trading System Trading System

  40. TSA MP E2E Automation Web based GUI (ISC based) Main features • Monitoring and problem analysis: Show… • Operational status, location, resource details • Actively show status changes • Problems with resources • Resource dependencies • Topology • Clusters and systems • Group membership trees • Operator instructions per resource • Automation logs per domain (cluster) • Filtered views … • matching a specified name filter • located on a specific system • with errors or warnings • with operator requests • Operational tasks • Activate/Deactivate Policy for an E2E domain • Start/Stop resources • Reset a resource from a non-recoverable error • Exclude/Include a node from the automation • Move a resource (selection in choice group)

  41. Automation Concepts - Resource States Each Resource has the following states: • Observed State – Describes the current runtime state of the resource. For example: Online, Offline, Starting, Stopping, … • Desired State – Describes the intended target state which is expressed within the automation policy or a result of requests. Possible values: Online or Offline • Operational State – Gives more details about the current automation processing on this resource. For example: StartRequestPending • Compound State – Provides a traffic-light indication to the operator if he has to react on some situation. Possible value: OK, Warning, Error, Fatal

  42. SA MP (base) TSA z/OS Browser Client Resource Types and Policy Elements • Reference Resource • is a reference to a resource in a cluster • Resource Group • Basic Group • Choice Group • Relationships • StartAfter, StopAfter, ForceDownBy SA MP (End-to-End Automation Component) CRM StartAfter SAP Ref DB2 Ref Adapter Adapter SAP DB2 SAP Cluster (AIX/pSeries with SA MP) Sysplex I (Sysplex with TSA z/OS)

  43. Architecture TSA AOAutomation Engine Database . Browser Client SA MP (Linux) Adapter Plug-infor TSA MP . . .. Daemon process . JMS SA MP (AIX) Adapter Plug-infor TSA MP . . .. . TSA. z/OS Adapter Plug-infor TSA z/OS . . .. Requests and events from TSA AO engine WebSphere Application Server 6 TSA AO Automation Access EJBs WebSphere Portal Server H Requests against TSA AO engine T T TSA forApplicationOperations GUI P S e r v TSA AOGeneric Adapter Integrated Solutions Console (ISC) e r TSA AOGeneric Adapter Data Warehouse TSA AOGeneric Adapter Common Event Infrastructure (CEI) TEC System Automation End-to-End Server

  44. SA MP (End-to-End Automation Component) -Cross Cluster Automation SA MP (End-to-End Automation Component) Adapter * New with SA MP 2.2 MSCS * New with SA MP 2.2 Adapter Adapter HACMP SA z/OS SA MP Adapter Adapter AIX SA MP z/OS on zSeries Adapter SA MP AIX SA MP Other HA cluster prod. Other HA cluster prod. Linux on PCs,BladeCenter Linux on zSeries Linux on POWER (p,i) Solaris HP-UX Windows PCs ( Windows / Linux ) UNIX ( AIX, Linux, Solaris, HP-UX ) zSeries ( Z/OS, zLinux)

  45. Policy with Resources, Groups and Relationships • A simple XML Document describes your IT Environment • Easy to edit – use any existing XML Editor • System Automation can deal with changes in your IT Environment. • Add Applications to Automation • Create new groups • Adapt relationships •  No Programming skill required ! <AutomationPolicy version="1.0"> <PolicyInformation> <PolicyName> End-to-End Demo Policy</PolicyName> <PolicyAuthor>Michael Atkins</PolicyAuthor> <PolicyDescription> This document contains a demo policy </PolicyDescription> </PolicyInformation> <ResourceReference name=“Ref A"> <DesiredState>Online</DesiredState> <Description> This is the reference to Application A </Description> <Owner>Bob Owens</Owner> <InfoLink>http://www.friendly.com/help/ApplA</InfoLink> <ReferencedResource> <AutomationDomain> AIX_Cluster </AutomationDomain> <Name>Appl A</Name> <Class>ResourceGroup</Class> </ReferencedResource> </ResourceReference ….. <Relationship> <Source> Ref A </Source> <Type> StartAfter </Type> <Target> DB Ref </Target> </Relationship> ….. <ResourceGroup name=“E2E_Resource_Group" > <DesiredState>Online</DesiredState> <Description> This is the group starting my Web-Service </Description> <Member> Ref A </Member> <Member>DB Ref</Member> </ResourceGroup> </AutomationPolicy> E2E_Resource_Group Ref A DB Ref

  46. What’s next on the Scope?

  47. Tivoli System Automation Roadmap

  48. Need More Information? Please contact me: Creighton Hicks hicksc@us.ibm.com

More Related