APV Technical Training

1. APV Technical Training Chapter 3 - Server Load Balancing

2. Objectives • To understand Server Load Balancing concepts. • To understand Array Server Load Balancing configuration steps.

3. Topics • Unit 1: Basic Load Balancing Concepts and Configuration • Unit 2: Advanced Load Balancing Examples • Appendix: Web Concepts

4. What is Server Load Balancing • SLB allows application to be scaled from a single server to a service farm. While distributing the load among available servers in the farm; SLB also doing content switching/routing to maximize available resources. • SLB Benefits • Scale Up Application Capacity – • Deploy additional servers quickly and transparently to end users. • Spread load among multiple servers improves response time to clients • Improve Application Availability – • Improved resiliency through Health Checking for server availability and automatically redistribute traffic to health server(s). • Cushion network overload, shield network attacks • Lower TCO by maximize network resource usage • Better control of service resource usage through implementation of different load balancing algorithms. • Traffic management based on TCP/UDP port number, URL, HTTP Host Header, Cookie content, or SSL Session ID

5. Unit 1: Basic SLB Concepts • Array SLB Terminology & Basic Configuration • SLB Group Methods • SLB Policy

6. Array SLB Terminology & Basic Configuration

7. Array SLB Basic Terminology • Real Service (“Real”) • Represents a physical service which host applications and handle client requests • SLB Group (“Group”) • Member: multiple Real Service Members which traffic will be distributed to • Method: the way to distribute traffic among Members • Virtual Service (“Virtual”) • Represents a virtual service that clients can connect to • SLB Policy (“Policy”) • Routing policy to dispatch traffic from Virtual Service to specific SLB Group

8. Basic SLB Configuration SLB Configuration Steps • Define Real Service(s) • Define IP address, service, maximum connections, and health check characteristics of the physical services that will be load balanced. • Configure SLB Group(s) • Create and select load balancing algorithm (method). • Add real services to a group. • Parameters (such as cookie value, URL string, or weight) may need to be specified when creating a group or adding real services to one. • Define Virtual Service(s) and Virtual Link • Define Virtual IP (VIP) address to front-end a group of services. • Define Virtual Link in case multiple policies are required to route request to a group • Configure SLB Policy • Bind Virtual Service to various SLB Group(s) with various policies.

9. SLB Real Service (Server, IP+port)

10. SLB Real Service • To define a real service • slb real <protocol><real_name> <real_ip|real_mac> [real_port] [max_conn] [hc_type] [hc_up] [hc_down] [timeout]

11. SLB Real Service (cont.) • Configuration Example • HTTP real service named “web1” with IP address “10.1.1.100” running on TCP port 80. • HTTP real service named “web2” with IP address “10.1.1.101” running on TCP port 80. • The real services can handle a maximum of 1500 connections. • Perform HTTP health checking requiring 2 successful health checks to bring the service online and 3 failed health checks to take it offline.

12. Manage Real Service - WebUI

13. Maintaining Real Services • Disabling real services for maintenance • To take a service out of service for maintenance (will not accept new connections) • slb real disable<real_name> • To put a service back into service after completing maintenance • slb real enable<real_name> • Verifying which services are disabled and enabled • To see which services are disabled • show slb real all

14. Real Service Activation • Array ADC appliance supports Real Service graceful/delayed activation • Recovery Time – time between “Active/Up” to start passing traffic. • Warm Up Time – Traffic ramping up time after start passing traffic to the server. • Examples: • Define Real Service “web1” 10 seconds Recovery Time, 60 seconds Warm Up time.

15. Real service Graceful Shutdown • Overview By default, if a real service is disabled, the APV appliance will not send session requests to it. However, for the real services using cookie-based group methods and load balancing polices, APV will still send the existing session requests that match the cookie to the disabled real service to ensure service persistence. The new session requests will be sent to other working real services. This is so-called “Graceful Shutdown”, as shown below.

16. Real service Graceful Shutdown (continue) • Configuration Example → Step 1 Disable the real service “r1”. → Step 2 Check the status of “r1”. “INACTIVE(waiting)” means it is still processing connection requests, i.e. it is in the process of “softclose”. → Step 3 After a while, check the status of “r1” again. “INACTIVE(suspend)” means the service has been shut down completely.

17. Real service Warm-up • Overview After a real service is enabled, it might receive a large number of requests immediately before it is ready. The sudden increase of traffic might lead to real service failure. To solve this problem, the APV appliance supports Warm-up activation of real services by setting recovery time and warm-up time for them. • New CLI Commands • slb real activation <real_name> <recovery_time> [warm-up_time] This command is used to set the recovery time and warm-up time for a real service. • no slb real activation <real_name> Remove the recovery and warm-up time settings of the specified real service. • show slb real activation <real_name> Display the recovery and warm-up time settings of the specified real service.

18. Real service Warm-up (continue) • Configuration Example → Step1 Configure warm-up activation for the real service “r1”. → Step 2 Enable the real service “r1”. → Step 3 Check the status of the real service “r1”. As shown in the above output, the status of “r1” during the recovery time is displayed as “UP (softup) ACTIVE”. No connection requests will be sent to a real service in “softup” status.

20. SLB Group (Server Pool)

21. SLB Group Definition • A SLB group allows requests to be distributed to a set of similar Real Services. • Example: 3 web services containing the same content. • The traffic shall be distributed based on least connection method • Two parts to defining a service group: • Group method selection (create a group) • Group member assignment • Member Weight • Member Priority

22. SLB Group – simple example • Configuration Example • Traffic should be distributed to group members using round robin • Group will contain real services named “web1” and “web2” • service “web2” should be allocated three times as many connections as service “web1”

23. SLB Group Method • Group Creation and Method Selection • slb group method<group_name> [method [other_parms]]

24. SLB Group Method (Load Balancing Algorithm)

25. SLB Group Methods • A group allows requests to be sent to a set of similar servers. • Example: 3 web servers (Real Services) containing the same content. • Group method determines how the load be distributed • Non-PersistentGroup Methods • Each new client connection may be distributed to a different real service within a group. • PersistentGroup Methods • All new connections from a particular client are sent to the same real service within a group. • Note: Real services that are Down/Disabled will not be used. Real a Group Real b Real c

26. Non-Persistent Methods Round Robin (rr) Least Connections (lc) Shortest Response time (sr) SNMP-Based (snmp) Persistent Methods (to a Server) IP-Based Persistent IP (pi) Hash IP (hi) Consistent Hash IP (chi) Hash IP and Port(hip) Header/Request-Based Hash Header (hh, chh) Persistent Hostname (ph) Persistent URL (pu) Hash Query (hq) SSL Session ID (sslsid) Cookie-Based Persistent Cookie (pc) Re-write Cookie (rc) Insert Cookie (ic) Hash Cookie (hc) Embedded Cookie(ec) Proximity SIP User ID/Caller ID SLB Group Methods • ArrayOS has a set of SLB Group Methods to spread load among Group Members (Server Farm).

27. Method - Round Robin (rr) • “rr” distributes new connections (requests) sequentially among available real services. • Example 1: Connections are spread to the real services in the following order: • A, B, C, A, B, C, A, B, C, …. • Real Service “weight” can be given. “rr” distributed new connections accord with the “weight”. • Example 2: Real A, B, C all with “weight” value of “3”, illustrates the distribution: • A, A, A, B, B, B, C, C, C, A, A, A, … • Example 3: give real A with “weight” 3. B and C are 1, The distribution - • A, A, A, B, C, A, A, A, B, C, etc. • To assigned Real Service weight; see • “slb group member <group> <real> [weight] Real A Real A W=3 Group (rr) Real B Real B W=3 Real B W=1 Real C Real C W=3 Real C W=1

28. Method - Least Connections (lc) • “lc” selects a Real Service with lowest active connection in a group for the new request (connection). • Example: Real C has the least active connections, so the new connection will go to Real C. • Real Service “weight” option is applicable to “lc” method. Real A Conn=100 Group Method = lc Real B Conn = 102 Real C Conn = 98 Real C Conn = 98

29. Method - Shortest Response Time (sr) • “sr” – select the Real Service in the group with the shortest response time. • Example: Real B current response time is 8ms; the shortest response among the severs. For new connection request; SLB will pick Real B. • The Response Time is based on – • L4 – TCP Setup time • L7 – Request/Response time Real A 10ms Group Method = lc Real B 8ms Real C Conn = 98

30. Method - SNMP • “SNMP” method is to select a Real Service based on the selected SNMP information retrieved from all Real Services (Servers). • For example: using SNMP method to select real server with the least CPU utilization. • SNMPv2 is used to retrieve backend server information. • OID is used to specify the server information to be retrieved. Real A CPU=16% Group Method = lc Real B CPU=10% Real C CPU=18%

31. Method - Proximity • “proximity” method uses GSLB proximity rules and HTTP redirect. It select the real server which has least proximity distance with the request IP. • For example: if client IP is with 71.132.x.x; Real B will ne selected • HTTP redirect to “www.a2.com” • If no match; first choice apply • See CLI – “slb group method <group_name> prox [rr|sr|lc] [threshold]” Real A 12.1.1.1 www.a1.com Group (proximity) Real B 71.132.1.8 www.a2.com Real C 70.1.1.1 www.a3.com

32. Internet Real Service Weighting (rr, lc) • Weighting allows you to change the distribution between real services in the group. • Example: You may set one service to handle twice as many connections as another service. • Use weighting if you have a mix of backend services with different performance capabilities. • Example: If you add new services that have faster CPUs, you could weight them higher so they receive more connections.

33. Server Persistent Concept Session - The requests/connections from the same client need be landed to the same backend server. 1 1 2 2 3 3

34. Persistent Group Methods • IP-Based • Persistent IP (pi) • Hash IP (hi) • Consistent Hash IP (chi) • Header/Request-Based • Hash Header (hh) • Persistent Hostname (ph) • Persistent URL (pu) • SSL Session ID (sslsid) • Cookie-Based • Persistent Cookie (pc) • Re-write Cookie (rc) • Insert Cookie (ic) • Hash Cookie (hc)

35. Persistent IP (pi) • Persistent IP sends all successive connections from a client IP address to the same Real Service. • Client sends a request to the virtual service. • Array hashes (mod 4003) source Client IP address of request to the selected real service which stored in the hash table entry. • In the hash table entry is empty (no Real Service selected), “First Choice” will be used to select a Real Service fill the hash table entry. • Array forwards all requests with the same IP address (IPs with the same hash value) to the same real service (store in the table entry). • Note • If a hash table entry without Real Service assigned or failed, new Real Service will be chosen (First Choice) and entered the hash table entry. • PI method may not result in the same real service being chosen across different appliances on failover. • The PI hash table can be flushed by “slb group flush …” which is useful when down/new Real Services would add

36. Hash IP (hi) • Distributes new connections to a Real Service based on a hash of the source IP address. • Client sends a request to the virtual service. • Array hashes source IP address of request to select a real service. • mod – number of available Real Service • Array forwards all requests with same hash value to the same real service. • Note • The hash table (size = # of Real Services) is filled with available Real Service, sorted by name. • If a Real Service fails, as the mod value changed/table size change, persistence will not be maintained for existing clients on healthy services. The same as Add/Delete Real Service. • Consistent mappings of client IP to real service across multiple Arrays during failover is not guaranteed.

37. Consistent Hash IP (chi) • Distributes new connections to a real service based on a hash of the source IP address. • Client sends a request to the virtual service. • Array hashes request source IP to select a real service (mod 4003). • Array forwards all requests with same hash value to the same real service. • Note • SLB Real Service Group is sorted by Real Service name. • Hash Table - ArrayOS fill up all hash entries (4003) from Real Service Group in turn. When Real Service Group change (add/delete), the Hash Table will be refilled. • When a Real Service fails (by Health Check), a health Real Service from the SLB Real Service Group will be selected. Hash Table persistence willbe maintained for existing clients on healthy services. • Mappings of client IP to Real Service are consistent across all Arrays so that clients will continue to go to the same real service on failover.

38. Hash IP and Port • Distributes new connections to a real server based on a hash of the source IP and the port • Client sends a request to the virtual service. • Array hashes source IP address + Port of request to select a real service. • Array forwards all requests with same hash value to the same real service. • Note • This is effective method for connections behind a mega-proxy to provide better load balancing than with source IP only methods.,

39. Hash Header (hh) • Distributes new connections to a real service based on a hash of a non-standard HTTP Header • Non-standard header = headers not defined in RFC 2616 (HTTP/1.1) • Example: • X-MSISDN header generated by WAP gateways • All wireless clients (mobile phones) proxied through a WAP gateway have the same source IP address. • Problem for client-IP persistence methods! • Options include cookie-persistence and Hash Header methods. • WAP gateway can receive MSISDN (phone number) of user from Radius and insert it into proxied HTTP request as X-MSISDN header. • Hash Header on this header provides good load distribution across the real services while maintaining client persistence.

40. Persistent Hostname (ph) • Distributes new connections to a real service based on a hash of the hostname found in the HTTP request’s Host header sent from the client. • Client sends a request to the virtual service with a Host: header. • Array examines Host: header value to select a real service. • Array forwards all requests with same Host: header value to the same real service.

41. Persistent URL (pu) • Distributes new connections to a real service based on a static match of a URL Tag and Value. • Client sends a request to the virtual service with a Tag and Value included in the URL. • Array hashes Value to select a real service. • Array forwards all requests with same Tag and Value to the same real service.

42. SSL Session ID (sslsid) • SSL Session ID (negotiated during SSL connection setup) is used to maintain a client-to-service binding. • Client opens an HTTPS connection to the virtual service. • Array selects a real service and forwards request to it. • Response from real service contains an SSL Session ID. • Array tracks SSL SID and real service combination. • Next request from client contains SSL SID. • Array examines SSL SID and sends request to the same real service that was chosen earlier. • May only be used when load balancing SSL connections.

43. Persistent Cookie (pc) • Real service is selected based on a static match of the cookie name/value pair. • Client sends a request to the virtual service. • Array selects a real service and forwards request to it. • Response from real service contains a specific cookie with a value denoting that service. • Client receives cookie in response. • Next request from client includes cookie. • Array examines cookie and sends request to proper real service. • Each real service within a group must be configured with a unique cookie value.

44. Rewrite Cookie (rc) • Array rewrites (modifies) a named cookie’s value in the service response. • Client sends a request to the virtual service. • Array selects a real service and forwards request to it. • Response from real service contains a specific cookie with a generic value. • Array rewrites cookie value based on the real service. • Client receives cookie in response. • Next request from client includes cookie. • Array examines cookie and sends request to indicated real service. • Used to simplify cookie based persistence configuration. • All backend services within a group must set the same name=value pair.

45. Insert Cookie (ic) • Array automatically inserts a Cookie in the service response • Client sends a request to the virtual service. • Array selects a real service and forwards request to it. • Response from real service does not contain a cookie, but cookie persistence is needed. • Array creates a cookie based on the real service and inserts it in the response. • Client receives cookie in response. • Next request from client includes cookie. • Array examines cookie, strips out cookie, and sends request to same real service as first request. • Used in cases where the web site does not have built-in cookie support.

46. Hash Cookie (hc) • Array ADC appliance Real Service selection is based on the hash (# of Real Services) of the specified cookie’s value. • Client initial request without matched session cookie; the request will be served by Default Group. Assume the Default Group servers will generate session cookie (i.e. JSESSIONID) to the client. • Client requests contain the session cookie (name and value). • Array hash the cookie value and select the Real Service for the request. Similar as PI (persistent IP). • All Real Servers need capable serve all client session when requested. • Client session is unique among all servers, i.e. maintained by a central database.

47. SLB Group Member (Real Service Pool)

48. SLB Group Member • Using “slb group member …” command to add Real Service into a SLB group. slb group member <group_name> <real_name> <[weight] | [string]> • Different weight can be assign to each Real Service to a group employing the Round Robin or Least Connection balancing methods. • If users want to assign a Real Service to a group employing the Persistent URL or Persistent Cookie balancing method, an associate “string” value must also be specified. To add a real service to a group balanced via PC or PU, use this command.

49. SLB Group Member Activation • Use “slb group activation ..” to set the max number of health servers in a server farm to serve production traffic • slb group activation <group_name> <num_of_rs> • AN(config)#show slb group activation group1 Group activation presetting: 1 real server priority active reason r1 1 NO HEALTH r2 2 NO Priority r3 3 YES

50. SLB Group Member • Group Member Assignment • slb group member <group_name> <real_name> [ <weight> | <param_string>]