Nginx Support Essentials for Apache Administrators

1. Nginx Support Essentials for Apache Administrators Chris Caillouet Sr. Linux Instructor chris.caillouet@rackspace.com Prepared for: Rackspace University Date: Oct 1, 2013 Rackspace Training Material

2. Created by and for Rackers! A special thanks to the Rackers who shared their time and knowledge to help build the materials and labs for this course: Ed Velez Herb Jackson Mike Filio James Dewey Bill Anderson Carl George Mike Hicklen Tavis Wilson John Schwinghammer Kevin Grigsby

3. Objective This course is designed to equip a seasoned Apache administrator with the essential skills necessary to provide system administration support on servers running the Nginx webserver – in an eight hour session. Throughout the course, we will leverage prior knowledge to teach the methods and tools necessary to perform the common tasks required to configure, secure, update and troubleshoot within this alternative Linux webserver. Topics will range from the basic differences in architecture, relevant commands, understanding of the common application configurations our customers utilize and providing the ability to perform typical support requests or maintenance tasks we encounter.

4. Introductions • Preferred Name • Role & Team • Shift • City • Tenure • Years of Linux? Red Hat? Debian? Other? • Favorite Rackspace Moment!

5. Rules of Engagement • Do as I say… wait, that sounds bad. I mean… while I am talking about a particular topic… try it out! We learn best by doing, so… DO try this during class! Lab exercises will follow each topic, but these will challenge you to use the basics you should pick up during instruction. • Target audience for this course is a seasoned Apache savvy Linux administrator. If you do not have that skill, go get it, then come back. We will leverage the information you already have with Apache to learn the tools needed to accomplish your daily tasks within Nginx based configurations. TL;DR: This is not an “Intro To…” course. • I am not an encyclopedic knowledge of all things Linux… yet. I will tackle as much as I can in the way of questions, but I am only human… for now. From time to time, you may have to google that $#!+ yourself. Fair warning.

6. Module I Major Differences

7. - No built in PHP support - Apache is process based while Nginx is event based - Static processing in Nginx to enhance performance - Modules in Nginx are compiled into the binary while Apache has installable packages for each module

8. - The Nginx configuration files are a series of includes for each site’s configuration whereas Apache has global settings and a companion virtual host with overrides for customization No "override" file like Apache's .htaccess - Consider the Nginx repos/sources for installation instead of standard OS repos/sources, to ensure latest versioning and feature/security set - Nginxhas a default user of 'nginx’

9. EXERCISES • Build a cloud server using a distro of your choice. • NOTE: I will be using a CentOS <latest version> for this purpose. • Open a browser and peruse wiki.nginx.org

10. Module II Installation

11. Nginx, as previously stated, is best installed from the Nginx repository/source instead of the native OS ones • Once you have this configured, Nginx can be installed, as expected • The main Nginx configuration file will be located at /etc/nginx/nginx.conf • There is also an included directory of /etc/nginx/conf.d where additional configuration files can be created • These files will need to be named ____.conf • Nginx has the ‘configtest’ verb defined in the init script and the ‘nginx –t’ command will perform the same check

12. Tour of the Nginx Configuration File /etc/nginx/nginx.conf access_log worker_processes listen worker_connections error _log location / { } http { } server { } gzip server_name index root

13. EXERCISES - Configure the nginx repository or source - Install nginx - Start the nginxservice - Ensure this service will start at boot time - Read the nginx.conf and man pages Hint: http://nginx.org/en/linux_packages.html

14. Module III File Based Configuration

15. - conf.d/domain.confconfig files start with a 'server { ’ block - lines are ended with a semicolon - No ‘ServerAlias’... all names go on the 'server_name' line - NOTE: This directive DOES take wildcards --->*.domain.com - Access Log location defined for a custom log file - location / = defines where resources will be located based on the incoming URL request - a location block can be written for each different handling need (e.g. mapping /blog to the proper subfolder) - root = path to the resources for that location - CAN be defined in the server { block, making it something similar to a true DocumentRoot - index = location specific DirectoryIndex file list

16. EXAMPLE FILE server { listen 80; server_namewww.example.com; location / { root /var/www/html; index index.htmlindex.htm; } }

17. EXERCISES Populate a simple index.html in the default location that serves the phrase, "Greetings Earth!" for example.com Create a new www.example.com configuration file that serves content from /var/www/html and resolves an index.html that displays the phrase 'www’ Create a new app.example.com configuration file that serves content from /var/www/vhosts/app with an index.html file that displays the phrase 'app'. Add password authentication for this last site for a simple user/pass request. (hint: http://wiki.nginx.org/HttpAuthBasicModule)

18. Module IV SSL Certificate Installation

19. This should feel quite familiar… • - necessary components to define in a conf.d/___.conf file: • listen 443; # OR consider "listen 443 ssl;" • ssl on; # instead of these two lines • ssl_certificate /path/to/cert.pem; #all certs in one file!!! • ssl_certificate_key /path/to/cert.key; • - this can be contained in it’s own file with a location block, or within the same configuration file as the HTTP (port 80) site

20. EXERCISES Hint - /etc/nginx/conf.d/example_ssl.conf - create a self-signed certificate using openssl for the www.example.comsite - enable SSL for the www.example.com site using this certificate. Try doing so within its own file. How about moving it into an existing configuration file? - Disable the SSLv2 protocol for the site

21. Module V Rewrite Rules

22. - Nginx supports Rewrite Rules using regex - http://wiki.nginx.org/IfIsEvil = IF IS EVIL!!!! Summary: best practice is to use try_files - In my opinion, it’s best to see examples

23. Redirect site path to another path - For a path /foo that should redirect to the path /bar we would use the following rule: location /foo { return 301 /bar; } location /foo { rewrite ^ http://domain.com/bar permanent; }

24. Redirect http traffic to https addresses ENTIRE SITE (inside server block for port 80) return 301 https://domain.com$request_uri FOR A SINGLE DIRECTORY WITHIN SITE [for all site requests to be sent to www.domain.com, use this same approach] location /path { return 301 https://domain.com$request_uri; }

25. EXERCISES - Configure a rule that sends all HTTP requests for www.example.com to the HTTPS URL. - Forward all requests for example.comto app.example.com

26. Module VI PHP & Proxies

27. PHP FastCGI • all PHP is handled outside of Nginx • You will need some type of PHP service implementation (php-fpm, uwsgi, or Apache's mod_php) • commented section in the main config file may not be ideal, as it uses tcpinstead of Unix sockets. (Note: Good choice for large numbers of connections, but not the basic setups.) • simply enabling the commented out areas is not enough... you must also configure a PHP service to handle the requests, such as php-fpm, as we will see…

28. PHP-FPM implementation - yum install, start, chkconfigphp-fpm (or php5-fpm, etc...) - enable the FastCGI directives found in the nginxdefault.conf - reference /etc/php-fpm.d/www.conf, for configuration details NOTABLE ADVICE: https://nealpoole.com/blog/2011/04/setting-up-php-fastcgi-and-nginx-dont-trust-the-tutorials-check-your-configuration/ - In short, within nginxdomain.conf(or default.conf), add the following inside of the location / { block: try_files$uri =404; #added as the first line in block

29. PROXIES - What is a proxy? - What are they used for? - Remember the ProxyPassdirective/setup in Apache - Why Proxy with nginx? - PHP, Django, cause_its_cool_like_dat, etc..

30. Customer uses Apache as a backend, so we need to pass that traffic over to the service EXAMPLE location ~ .*\\.(php)$ { proxy_pass http://127.0.0.1:8080; proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; client_max_body_size 10m; client_body_buffer_size 128k; proxy_connect_timeout 90; proxy_send_timeout 90; proxy_read_timeout 90; proxy_buffer_size 4k; proxy_buffers 4 32k; proxy_busy_buffers_size 64k; proxy_temp_file_write_size 64k; }

31. pointing to a specific proxy server/service for PHP request is implemented within the default/domain.conffile • minimum config is commented out in default.conf --> uncomment/copy & restart Example Apache proxy setup: https://one.rackspace.com/display/MGC/Reverse+Proxying

32. EXAMPLE WITH MULTIPLE BACKENDS upstream backends { server backend1.fqdn:port; server backend2.fqdn:port; } location ~ .*\\.(php)$ { proxy_pass http://backends; proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; client_max_body_size 10m; client_body_buffer_size 128k; proxy_connect_timeout 90; proxy_send_timeout 90; proxy_read_timeout 90; proxy_buffer_size 4k; proxy_buffers 4 32k; proxy_busy_buffers_size 64k; proxy_temp_file_write_size 64k; }

33. EXERCISES - Implement php-fpm for the app.example.comsite - Set the pm.max_children to a value of 75 for PHP-FPM - Have the appropriate content displayed for app.example.com/info.php HINT: configured in /etc/php-fpm.d/www.conf ALSO: fastcgi_param SCRIPT_FILENAME /PATH/TO/ROOT$fastcgi_script_name;

34. Module VII Modules

35. - Apache allows us to simply install modules via packages - Not the case with nginx - but many are already installed - Nginx must be recompiled to add any new "module" features - compile from source steps including full path to new module's file (potentially the delineation of SoS due to the fact these will not be updated and introduce a security concern) - most modules we need are pre-compiled into the nginx package and simply need to be enabled

36. EXERCISES - Configure the location app.example.com/status to display the appropriate status information using a module available to nginx [status module] - Attempt to recompile nginx to add the set_miscmodule HINT1: You may need to disable your previous rewrite rules! HINT 2: [Mis]information - http://wiki.nginx.org/HttpStubStatusModule

37. Module IX Tuning & Optimization

38. Nginx Configuration Settings • worker_processes- setting this to 'auto' will have it scale as your server grows • - worker_connections - MaxClients calculation in nginx is worker_process x worker_connections • TUNE IF... connection refused, but still have resources available. Keep a close watch on number of open file descriptors set in the OS, as nginx can end up scaling beyond that limit. Adjust using ulimit tool to understand/adjust the max open file descriptor value and within nginx set the parameter 'worker_rlimit_nofile' which specifies the number of file descriptors nginx can have open at any given time (should not exceed ulimit setting)

39. Nginx Configuration Settings - tcp_nodelay = default is off TUNE IF... you are sending large volumes of small traffic in your environment. By default it will wait to send larger packets and this may not be ideal for the customers use case. - gzip = off by default. TUNE IF... you need to enable gzip compression on sent data to reduce size of traffic.

40. Nginx Configuration Settings In Depth GZip Example gzipon; gzip_min_length 10240; gzip_proxied expired no-cache no-store private auth; gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml; gzip_disable "MSIE [1-6]\.";

41. Nginx Configuration Settings - Consider disabling the access logs, if not needed: access_log off; TUNE IF... customer requested this when leveraging 3rd party analytics via additional code in their site.

42. KERNEL LEVEL ADJUSTMENTS • These settings could be necessary changes for high volume sites • Configure kernel paramatersusing 'sysctl’ • Update the same parameters within the /etc/sysctl.confto make these updates permanent

43. # Increase system IP port limits to allow for more connections net.ipv4.ip_local_port_range = 2000 65000 net.ipv4.tcp_window_scaling = 1 # number of packets to keep in backlog before the kernel starts dropping them net.ipv4.tcp_max_syn_backlog = 3240000 # increase socket listen backlog net.core.somaxconn = 3240000 net.ipv4.tcp_max_tw_buckets = 1440000 # Increase TCP buffer sizes net.core.rmem_default = 8388608 net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216 net.ipv4.tcp_congestion_control = cubic

44. EXERCISES HINT 1: Nginx Configuration Changes • - Inspect the worker_processes and worker_connections setting in the nginx configuration file • - Set worker_processes to auto • - Increase the worker_connections to 2048 • - inspect the limit for max open files • set the system limit for max open files to 2048 • - Inspect the default value for port ranges • - set the port range for connections to be restricted to the ports between 2000 and 5000 • NOTE – MAKE ALL CHANGES PERSIST A REBOOT HINT 2: ulimit changes HINT 3: sysctl changes

45. EXTRA MATERIALS Rackspace Knowledge  http://www.rackspace.com/knowledge_center/ https://one.rackspace.com/display/MGC/nginx+Portal Forums Galorums!  wiki.nginx.org Helpful Links serverfault.com failverse.com (A Racker’s Blog) rtcamp.com/wordpress-nginx/ AND OF COURSE… https://www.google.com/

46. fin