Enhancing Web Server Scalability with Linux Event-Dispatch Mechanisms
230 likes | 375 Vues
This document explores the scalability of web servers using event-based mechanisms in Linux, addressing the challenges posed by heavy traffic from numerous clients across WANs. It examines the issues of concurrent connections, high latency, and the need for high throughput under significant loads. The paper evaluates different Linux event-dispatch mechanisms, including select(), poll(), and real-time signals. It also introduces improvements such as signal-per-fd to optimize resource allocation and handling of network events, aiming to enhance server performance and responsiveness.
Enhancing Web Server Scalability with Linux Event-Dispatch Mechanisms
E N D
Presentation Transcript
Scalability of Linux Event-Dispatch Mechanisms David Mosberger Hewlett Packard Labs Palo Alto Abhishek Chandra University of Massachusetts Amherst
Motivation Web Server Clients WAN • Large Web and Internet traffic • Heavily Loaded/Accessed Web Servers • cnn.com, britneyspearsfans.com, … • Starr Report, Napster ruling, ... • Challenge: Make Web Servers Scalable
Server Scalability Issues • Large number of concurrent/idle connections • Last-mile problem: Slow end-connections • High latency WAN traffic • HTTP/1.1 Persistent Connections • Heavy Request Loads • Need for high throughput • Pure Thread-based vs. Event-based servers • Focus: Scalability of Event-based servers on Linux
Outline • Motivation • Event-based Servers • Linux Event-Dispatch Mechanisms • Evaluation: Handling concurrent connections • RT signals and Signal-per-fd enhancement • Evaluation: Handling request load • Concluding Remarks
1. Interest Set Specification 3. Event Notification 4. I/O Handling Interest Set 2. Network Event Event-Based Servers • Server specifies Interest Set to Kernel • Kernel notifies Server of Event on a connection • Server handles I/O on the connection Server Kernel Connections
Linux Event-Dispatch Mechanisms • select() system call • poll() system call • /dev/poll interface • POSIX.4 Real-Time Signals
Interest Set Ready Set Scan select() system call • Interest Set specified on each call • Notification requires scan of interest set Server Kernel Connections
poll() and /dev/poll • Interest Set: • List of pollfd structures • Better for sparse interest sets, worse for dense sets • Notification • Requires scan of Interest Set • /dev/poll: • Interest Set specified incrementally • More compact ready set
POSIX.4 Real Time Signals • RT signals are queued • Multiple signals of same type can be delivered • RT signals carry a data payload (siginfo) • Provides the context of the signal • sigwaitinfo() system call: • Dequeues signals • Avoids overhead of calling signal handler • Signal can be blocked
1. Associate RT Signal 4.sigwaitinfo() 3. EnQueue Signal 5. DeQueue Signal 2. Network Event Using Real Time Signals for Network I/O • Interest Set specified incrementally • No scanning of Interest Set required Server Kernel Signal Queue Interest Set Sockets
Outline • Motivation • Event-based Servers • Linux Event-Dispatch Mechanisms • Evaluation: Handling concurrent connections • RT signals and Signal-per-fd enhancement • Evaluation: Handling request load • Concluding Remarks
Evaluation: Handling Concurrent Connections • Dispatch overhead and latency as a function of number of concurrent connections • Experimental Setup • 400 MHz P3 Linux 2.4.0-test7 server • μ -server using select(),/dev/poll or RT signals • 10 clients running httperf • Fixed request rate, increasing number of connections
Server CPU Usage 500 req/s • RT signal overhead independent of no. of concurrent connections
Response Time 500 req/s • RT signal response time independent of no. of concurrent connections
Drop 3 Network Event Network Event Network Event Limitations of Real Time Signals • Signal Queue Overflow: • New events lost • Can lead to “hung server” • Unfair Allocation of Signal Queue Server 3 3 2 Kernel Signal Queue Interest Set Sockets 1 2 3 4
Handling Signal Queue Overflow • Fallback mechanism • select(), poll(), etc. • Reconstruct current state • Issues • Server complexity • Overhead of maintaining explicit interest sets • Potential performance penalty
1 Discard Network Event Network Event Network Event RT Signal Enhancement:Signal-per-fd • Goals: • Avoid signal queue overflows • Fair Allocation of signal queue • Solution: Enqueue only one signal per socket Server 3 2 Kernel Signal Queue Interest Set Sockets 1 2 3 4
Signal-per-fd • Idea: • Signal queue length same as fdset size • Bitmap used to efficiently determine presence/absence of signal in queue • Advantages: • Simpler Server Implementation • No signal queue overflows • No need for fallback mechanisms • Fair Allocation of Signal Queue Resource • Avoids too fine-grained event notification • Coalesce multiple events for a socket
Outline • Motivation • Event-based Servers • Linux Event-Dispatch Mechanisms • Evaluation: Handling concurrent connections • RT signals and Signal-per-fd enhancement • Evaluation: Handling request load • Concluding Remarks
Server Throughput 6000 idle connections • Linear scaling of RT signals, signal-per-fd
Server CPU Usage 6000 idle connections • Linear Scaling of RT signals, signal-per-fd
Related Work • Event-Delivery API [BMD99] • Performance studies: • select() [BM98], /dev/poll [PL00] • RT signals [PLT00] • Web Servers: • Event-based: Flash [PDZ99], phhttpd [Brown99] • In-kernel: TUX, khttpd, AFPA [JKNRT01] • Future: Linux 2.5 Asynchronous I/O?
Summary • Scalability issues with Linux Event-dispatch mechanisms • Real Time Signals are scalable • Performance independent of number of concurrent connections • Signal Queue Overflow Problems • Signal-per-fd enhancement • potentially improves performance • reduces server complexity • provides fairness • Patch available at http://www.netli.com/links
