LegalTech® New York January 29 – 31, 2013 Protecting Your Crown Jewels in a Virtual World: Strategic Systems Recovery January 29, 2013 3:45 - 5 PMNew York Hilton
Speakers: Marcus Bluestein, Kraft Kennedy, Moderator Steven Shock, Irell & Manella LLP Jim McCue, Rodey Law Firm Steve Skidmore, Martin, Clearwater & Bell LLP
Industry Trends Marcus Bluestein Chief Technology Officer Kraft Kennedy firstname.lastname@example.org
Industry Trends Use of server virtualization increase again…
Industry Trends …as does use of SANs.
Industry Trends Traditional backup systems are declining
Industry Trends Firms increase use of disk-based backup
Industry Trends …increase use of replication technology,
Industry Trends …and slowly begin to explore online backup.
RPO and RTO Recovery Point Objective (RPO) The amount of data loss associated with a service failure. The Recovery Point Objective (RPO) is the point in time to which you must recover data as defined by your organization. This is generally a definition of what an organization determines is an "acceptable loss" in a disaster situation. If the RPO of a company is 2 hours and the time it takes to get the data back into production is 5 hours, the RPO is still 2 hours. Based on this RPO the data must be restored to within 2 hours of the disaster. Recovery Time Objective (RTO) The amount of downtime associated with a service failure. This is meant to describe the amount of time between when a service becomes unavailable to users and when service is restored. This includes time to try to fix the problem without resorting to the recovery procedures, the time to enact the recovery process, testing of the recovered system and communication to the users.
RPO and RTO Applications
Strategic Disaster Recovery with Exchange 2010 Steven Shock Chief Technology Officer Irell & Manella LLP SShock@irell.com (310) 203-7511
Irell & Manella LLP 450 users 2 Offices IP Litigation Concierge Service Retention Policy
Irell Environment • Los Angeles Primary Data Center • Newport Beach Data Center • Arizona Data Center • HP Server – DL 380: 20 Hosts • HP Storage – HP P4000 Lefthand: 1.5 Petabyte raw storage • VMWare 5.1 - SRM • Cisco Nexus Core Switching • MPLS – 100mbps each location • Watchguard Branch Office VPN – LA & AZ
Client Connections MAPI Connects through the CAS Role What “Mailbox” Server is listed in Outlook Profile?
Database Availability Group Replication Boundary High Availability vs. Site Resiliency HA is Site-based Site Resiliency Across Sites Consider Recovery Objectives Quorum Isolation
Redundancies Servers / VMs Load Balancing Multiple Database Copies Shadow Redundancy
Design Philosophy New Design Philosophy Recovery Objectives Database Design Dependent on Environment Simple Complex Moderately Complex Staggering Number of Options!
Design Options Too Many Options! • High Availability • Exchange Native • VMware • Third Party On-premise • Third Party Email Continuity • Client Connectivity • Outlook AnyWhere • Outlook WebApp • SSLVPN / UAG • Mobile Devices • MDM • Replication • Exchange Native • Third Party Software • SAN-based • Future Integration • Disaster Recovery • Exchange Native • Third Party Email Continuity • Third Party On-premise • Software Based • SAN Solutions
Recovery Objectives Reasonable Objectives are Usually Exceeded Don’t Increase Firm Expectations! What Failures are you Designing for? Failure Domain(s) Time Objective (RTO) Point Objective (RPO)
Database Design Philosophies: Old and New Policy Backups Recovery Objectives Large Databases Archive Database Lagged Copies
Database Single Item Recovery Recovery Database Override Mount Dial Offline Database Copy Seeding http://blogs.technet.com/b/timmcmic/archive/2011/07/12/exchange-2010-using-vss-to-perform-an-online-offline-database-seed.aspx
CAS / Hub Roles Multiple Servers Hosting Roles Production and DR NLB for CAS Hub – Native HA
Network / WAN WAN Redundancy Links Carriers Equipment Replication Network DAG Replication Maximum Latency Adjust Cluster Settings?
Other Options HA at Virtualization Level SAN Replication Database or Full Environment Managed & Cloud Services
DR Testing Worse Than a Disaster? Full Failover vs. Unit Testing VMware / Site Recovery Manager Restore Environment in a “Bubble” Restoring DAG Can be Difficult Consider New Server and Database Portability
Site Failover Database Failover Activation Blocking CAS Proxying Site Failover Start/Restore DAG Update DNS Internet Mail
Site Failback Restore / Resume Servers Resume / Update Database Copies Verify Health Failover Databases Update DNS
Lessons Learned - 1 NLB / Kerberos Authentication Consider DR site http://technet.microsoft.com/en-us/library/ff808312.aspx DAG in a Single Site? Put DAG in DAC Mode Anyway DatacenterActivationMode [DAGName] –DatacenterActivationMode DAGOnly Only One CAS Server? Still Consider using CASArray Setting Future Growth
Lessons Learned - 2 Failback URL http://blogs.technet.com/b/mbaher/archive/2010/11/27/robert-s-rules-of-exchange-namespace-planning.aspx Namespace Considerations Internet Name(s) MAPI Don’t Mix!
Steven Shock Chief Technology Officer Irell & Manella LLP SShock@irell.com (310) 203-7511
Back Up/Recovery with VMWare Jim McCue Information Systems Manager Rodey Law Firm email@example.com
Rodey Environment Overview 75 attorneys/150 people/2 offices Litigation/Business 30 mb QMOE WAN Albuquerque – Santa Fe
Rodey Environment Virtualization 80% virtual 2 datacenters (Albuquerque, Santa Fe) 4 VMWare 4.1 esxi hosts w H/A 30 Guests Exchange 2010 Aderant Autonomy BES, File Servers, DCs, MailMarshal
Rodey Environment EqualLogic SANS Albuquerque 2 x PS6000XV 13 TB usable 3.5 TB data + replication Santa Fe PS6000E 11 TB usable 3 TB data + replication Local disk backup storage Albuquerque, Santa Fe Drobo B1200i: 12TB Disk to tape: Dell Powervault lto5 tape autoloader
Rodey Goals RPO (Recovery Point Objective)=4 hour RTO (Recovery Time Objective)=8 hours, 16 hours Tier one applications=8 hours Network Authentication E-Mail Documents Time Entry & Billing Tier two applications=16 hours E-discovery & Litigation DBs BES Citrix
Rodey Goals D/R Move operations to other office Replication Restores VM Level Recovery Application Level Recovery Files, SQL, Exchange, A/D Archive to tape->move to disk offsite in the future
EqualLogic Thin provisioning Snapshots Replication 6.0 firmware: Synchronous EqualLogic Snapshot Manager/VMware Edition (ASM/VE) VMWare Tools quiescence VMWare consistent image EqualLogic Snapshot Manager/Microsoft Edition (ASM/ME) Volume Shadow Copy: Exchange, SQL Requires guest attached storage Transactional consistent image No log truncation
Veeam Backup & Replication Bare metal hypervisor backup Vmware/Hyper-V Deduplication Compression Changed Block Tracking (VMware CBT)
Veeam Backup & Replication Application aware processing Exchange, SQL, AD, Sharepoint Veeam agent -> MS VSS agents No permanent guest agent Transactional consistent image Log truncation
Veeam Backups Full, Incremental Reverse Incremental
Veeam Replication Replication Full + VMWare snapshots Network mapping and re-IP
Veeam Replication Replication Recovery Failover Failback
Veeam Verification SureBackup Live Verification Virtual Lab Application Group Scheduled
Veeam Verification Application Group - Role