C++ Network Programming Mastering Complexity with ACE & Patterns

Dr. Douglas C. Schmidt d.schmidt@vanderbilt.edu www.cs.wustl.edu/~schmidt/tutorials-ace.html C++ Network ProgrammingMastering Complexity with ACE & Patterns Professor of EECS Vanderbilt University Nashville, Tennessee

Motivation: Challenges of Networked Applications Complexities in networked applications • Accidental Complexities • Low-level APIs • Poor debugging tools • Algorithmic decomposition • Continuous re-invention/discovery of core concepts & components • Inherent Complexities • Latency • Reliability • Load balancing • Causal ordering • Scheduling & synchronization • Deadlock • Observation • Building robust, efficient, & extensible concurrent & networked applications is hard • e.g., we must address many complex topics that are less problematic for non-concurrent, stand-alone applications

Presentation Outline • Presentation Organization • Background • Concurrent & network challenges & solution approaches • Patterns & wrapper facades in ACE + applications Cover OO techniques & language features that enhance software quality • Patterns (25+),which embody reusable software architectures & designs • ACE wrapper facades, which encapsulate OS concurrency & network programming APIs • OO language features, e.g., classes, dynamic binding & inheritance, parameterized types

The Evolution of Information Technologies CPUs and networks have increased by 3-7 orders of magnitude in the past decade 2,400 bits/sec to 1 Gigabits/sec These advances stem largely from standardizing hardware & software APIs and protocols, e.g.: 10 Megahertz to 1 Gigahertz • Intel x86 & Power PC chipsets • TCP/IP, ATM Increasing software productivity and QoS depends heavily on COTS • POSIX & JVMs • Middleware & components • Quality of service aspects • Extrapolating this trend to 2010 yields • ~100 Gigahertz desktops • ~100 Gigabits/sec LANs • ~100 Megabits/sec wireless • ~10 Terabits/sec Internet backbone In general, software has not improved as rapidly or as effectively as hardware

Component Middleware Layers Historically, mission-critical apps were built directly atop hardware & OS • Tedious, error-prone, & costly over lifecycles There are layers of middleware, just like there are layers of networking protocols • Standards-based COTS middleware helps: • Control end-to-end resources & QoS • Leverage hardware & software technology advances • Evolve to new environments & requirements • Provide a wide array of reuseable, off-the-shelf developer-oriented services There are multiple COTS layers & research/ business opportunities

Operating System & Protocols INTERNETWORKING ARCH MIDDLEWARE ARCH RTP TFTP FTP HTTP Middleware Applications DNS TELNET Middleware Services UDP TCP IP Middleware Fibre Channel Solaris VxWorks Ethernet ATM FDDI Win2K Linux LynxOS 20th Century 21st Century • Operating systems & protocols provide mechanisms to manage endsystem resources, e.g., • CPU scheduling & dispatching • Virtual memory management • Secondary storage, persistence, & file systems • Local & remove interprocess communication (IPC) • OS examples • UNIX/Linux, Windows, VxWorks, QNX, etc. • Protocol examples • TCP, UDP, IP, SCTP, RTP, etc.

Host Infrastructure Middleware Asynchronous Event Handling • Examples • Java Virtual Machine (JVM), Common Language Runtime (CLR), ADAPTIVE Communication Environment (ACE) Asynchronous Transfer of Control Physical Memory Access Synchronization Memory Management Scheduling www.cs.wustl.edu/~schmidt/ACE.html www.rtj.org • Host infrastructure middleware encapsulates & enhances native OS mechanisms to create reusable network programming components • These components abstract away many tedious & error-prone aspects of low-level OS APIs Domain-Specific Services Common Middleware Services Distribution Middleware Host Infrastructure Middleware

Distribution Middleware • Examples • OMG CORBA, Sun’s Remote Method Invocation (RMI), Microsoft’s Distributed Component Object Model (DCOM) • Distribution middleware avoids hard-coding client & server application dependencies on object location, language, OS, protocols, & hardware • Distribution middleware defines higher-level distributed programming models whose reusable APIs & components automate & extend native OS capabilities Domain-Specific Services Common Middleware Services Distribution Middleware Host Infrastructure Middleware

Common Middleware Services • Examples • CORBA Component Model & Object Services, Sun’s J2EE, Microsoft’s .NET • Common middleware services support many recurring distributed system capabilities, e.g., • Transactional behavior • Authentication & authorization, • Database connection pooling & concurrency control • Active replication • Dynamic resource management • Common middleware services augment distribution middleware by defining higher-level domain-independent services that focus on programming “business logic” Domain-Specific Services Common Middleware Services Distribution Middleware Host Infrastructure Middleware

Domain-Specific Middleware • Examples • Siemens MEDSyngo • Common software platform for distributed electronic medical systems • Used by all ~13 Siemens MED business units worldwide Modalities e.g., MRI, CT, CR, Ultrasound, etc. • Boeing Bold Stroke • Common software platform for Boeing avionics mission computing systems • Domain-specific middleware services are tailored to the requirements of particular domains, such as telecom, e-commerce, health care, process automation, or aerospace Domain-Specific Services Common Middleware Services Distribution Middleware Host Infrastructure Middleware

Overview of Patterns • Present solutions to common software problems arising within a certain context • Help resolve key software design forces • Flexibility • Extensibility • Dependability • Predictability • Scalability • Efficiency Client Proxy Service AbstractService service service service • Capture recurring structures & dynamics among software participants to facilitate reuse of successful designs • Generally codify expert knowledge of design strategies, constraints & “best practices” 1 1 The Proxy Pattern

Overview of Pattern Languages • Motivation • Individual patterns & pattern catalogs are insufficient • Software modeling methods & tools that just illustrate how, not why, systems are designed • Benefits of Pattern Languages • Define a vocabulary for talking about software development problems • Provide a process for the orderly resolution of these problems • Help to generate & reuse software architectures

Taxonomy of Patterns & Idioms

The Layered Architecture of ACE www.cs.wustl.edu/~schmidt/ACE.html • Features • Open-source • 200,000+ lines of C++ • 40+ person-years of effort • Ported to many OS platforms • Large open-source user community • www.cs.wustl.edu/~schmidt/ACE-users.html • Commercial support by Riverace • www.riverace.com/

Sidebar: Platforms Supported by ACE • ACE runs on a wide range of operating systems, including: • PCs, e.g., Windows (all 32/64-bit versions), WinCE; Redhat, Debian, and SuSE Linux; & Macintosh OS X; • Most versions of UNIX, e.g., SunOS 4.x and Solaris, SGI IRIX, HP-UX, Digital UNIX (Compaq Tru64), AIX, DG/UX, SCO OpenServer, UnixWare, NetBSD, & FreeBSD; • Real-time operating systems, e.g., VxWorks, OS/9, Chorus, LynxOS, Pharlap TNT, QNX Neutrino and RTP, RTEMS, & pSoS; • Large enterprise systems, e.g., OpenVMS, MVS OpenEdition, Tandem NonStop-UX, & Cray UNICOS • ACE can be used with all of the major C++ compilers on these platforms • The ACE Web site at http://www.cs.wustl.edu/~schmidt/ACE.html contains a complete, up-to-date list of platforms, along with instructions for downloading & building ACE

Key Capabilities Provided by ACE Event Handling & IPC Service Access & Control Synchronization Concurrency

The Pattern Language for ACE • Pattern Benefits • Preserve crucial design information used by applications & middleware frameworks & components • Facilitate reuse of proven software designs & architectures • Guide design choices for application developers

POSA2 Pattern Abstracts Service Access & Configuration Patterns The Wrapper Facade design pattern encapsulates the functions and data provided by existing non-object-oriented APIs within more concise, robust, portable, maintainable, and cohesive object-oriented class interfaces. The Component Configurator design pattern allows an application to link and unlink its component implementations at run-time without having to modify, recompile, or statically relink the application. Component Configurator further supports the reconfiguration of components into different application processes without having to shut down and re-start running processes. The Interceptor architectural pattern allows services to be added transparently to a framework and triggered automatically when certain events occur. The Extension Interface design pattern allows multiple interfaces to be exported by a component, to prevent bloating of interfaces and breaking of client code when developers extend or modify the functionality of the component. Event Handling Patterns The Reactor architectural pattern allows event-driven applications to demultiplex and dispatch service requests that are delivered to an application from one or more clients. The Proactor architectural pattern allows event-driven applications to efficiently demultiplex and dispatch service requests triggered by the completion of asynchronous operations, to achieve the performance benefits of concurrency without incurring certain of its liabilities. The Asynchronous Completion Token design pattern allows an application to demultiplex and process efficiently the responses of asynchronous operations it invokes on services. The Acceptor-Connector design pattern decouples the connection and initialization of cooperating peer services in a networked system from the processing performed by the peer services after they are connected and initialized.

POSA2 Pattern Abstracts (cont’d) Synchronization Patterns The Scoped Locking C++ idiom ensures that a lock is acquired when control enters a scope and released automatically when control leaves the scope, regardless of the return path from the scope. The Strategized Locking design pattern parameterizes synchronization mechanisms that protect a component’s critical sections from concurrent access. The Thread-Safe Interface design pattern minimizes locking overhead and ensures that intra-component method calls do not incur ‘self-deadlock’ by trying to reacquire a lock that is held by the component already. The Double-Checked Locking Optimization design pattern reduces contention and synchronization overhead whenever critical sections of code must acquire locks in a thread-safe manner just once during program execution. Concurrency Patterns The Active Object design pattern decouples method execution from method invocation to enhance concurrency and simplify synchronized access to objects that reside in their own threads of control. The Monitor Object design pattern synchronizes concurrent method execution to ensure that only one method at a time runs within an object. It also allows an object’s methods to cooperatively schedule their execution sequences. The Half-Sync/Half-Async architectural pattern decouples asynchronous and synchronous service processing in concurrent systems, to simplify programming without unduly reducing performance. The pattern introduces two intercommunicating layers, one for asynchronous and one for synchronous service processing. The Leader/Followers architectural pattern provides an efficient concurrency model where multiple threads take turns sharing a set of event sources in order to detect, demultiplex, dispatch, and process service requests that occur on the event sources. The Thread-Specific Storage design pattern allows multiple threads to use one ‘logically global’ access point to retrieve an object that is local to a thread, without incurring locking overhead on each object access.

The Frameworks in ACE

Example: Applying ACE in Real-time Avionics Key Results • Test flown at China Lake NAWS by Boeing OSAT II ‘98, funded by OS-JTF • www.cs.wustl.edu/~schmidt/TAO-boeing.html • Also used on SOFIA project by Raytheon • sofia.arc.nasa.gov • First use of RT CORBA in mission computing • Drove Real-time CORBA standardization • Goals • Apply COTS & open systems to mission-critical real-time avionics • Key System Characteristics • Deterministic & statistical deadlines • ~20 Hz • Low latency & jitter • ~250 usecs • Periodic & aperiodic processing • Complex dependencies • Continuous platform upgrades

Example: Applying ACE to Time-Critical Targets • Goals • Detect, identify, track, & destroy time-critical targets Challenges are also relevant to TBMD & NMD Key System Characteristics • Real-time mission-critical sensor-to-shooter needs • Highly dynamic QoS requirements & environmental conditions • Multi-service & asset coordination Key Solution Characteristics • Efficient & scalable • Affordable & flexible • COTS-based • Adaptive & reflective • High confidence • Safety critical • Time-critical targets require immediate response because: • They pose a clear and present danger to friendly forces & • Are highly lucrative, fleeting targets of opportunity

Example: Applying ACE to Large-scale Routers BSE IOM BSE BSE IOM IOM IOM IOM IOM IOM BSE BSE BSE IOM IOM IOM IOM IOM IOM BSE BSE BSE IOM IOM IOM IOM IOM Key Software Solution Characteristics • High confidence & scalable computing architecture • Networked embedded processors • Distribution middleware • FT & load sharing • Distributed & layered resource management • Affordable, flexible, & COTS • Goal • Switch ATM cells + IP packets at terabit rates • Key System Characteristics • Very high-speed WDM links • 102/103 line cards • Stringent requirements for availability • Multi-layer load balancing, e.g.: • Layer 3+4 • Layer 5 www.arl.wustl.edu

Example: Applying ACE to Hot Rolling Mills www.siroll.de Key Software Solution Characteristics • Affordable, flexible, & COTS • Product-line architecture • Design guided by patterns & frameworks • Windows NT/2000 • Real-time CORBA (ACE+TAO) • Goals • Control the processing of molten steel moving through a hot rolling mill in real-time • System Characteristics • Hard real-time process automation requirements • i.e., 250 ms real-time cycles • System acquires values representing plant’s current state, tracks material flow, calculates new settings for the rolls & devices, & submits new settings back to plant

Example: Applying ACE to Real-time Image Processing Key Software Solution Characteristics • Affordable, flexible, & COTS • Embedded Linux (Lem) • Compact PCI bus + Celeron processors • Remote booted by DHCP/TFTP • Real-time CORBA (ACE+TAO) www.krones.com • Goals • Examine glass bottles for defects in real-time • System Characteristics • Process 20 bottles per sec • i.e., ~50 msec per bottle • Networked configuration • ~10 cameras

Networked Logging Service Example • Key Participants • Client application processes • Generate log records • Server logging daemon • Receive, process, & store log records • The logging server example in C++NPv2 is more sophisticated than the one in C++NPv1 • C++ code for all logging service examples are in • ACE_ROOT/examples/ C++NPv1/ • ACE_ROOT/examples/ C++NPv2/ • There’s an extra daemon involved

Patterns in the Networked Logging Service Leader/ Followers Monitor Object Active Object Half-Sync/ Half-Async Reactor Pipes & Filters Acceptor- Connector Component Configurator Proactor Wrapper Facade Thread-safe Interface Strategized Locking Scoped Locking

Network Daemon Design Dimensions • Communication dimensions address the rules, form, & level of abstraction that networked applications use to interact • Concurrency dimensions address the policies & mechanisms governing the proper use of processes & threads to represent multiple service instances, as well as how each service instance may use multiple threads internally • Service dimensions address key properties of a networked application service, such as the duration & structure of each service instance • Configuration dimensions address how networked services are identified & the time at which they are bound together to form complete applications

Communication Design Dimensions • Communication is fundamental to networked application design • The next three slides present a domain analysis of communication design dimensions, which address the rules, form, and levels of abstraction that networked applications use to interact with each other • We cover the following communication design dimensions: • Connectionless versus connection-oriented protocols • Synchronous versus asynchronous message exchange • Message-passing versus shared memory

Connectionless vs. Connection-oriented Protocols • A protocol is a set of rules that specify how control & data information is exchanged between communicating entities SYN SYN/ACK ACK Acceptor Connector 3-way handshake in TCP/IP • Connection-oriented applications must address two additional design issues: • Data framing strategies, e.g., bytestream vs. message-oriented • Connection multiplexing (muxing) strategies, e.g., multiplexed vs. nonmultiplexed

Alternative Connection Muxing Strategies • In multiplexed connections all client requests emanating from threads in a single process pass through one TCP connection to a server process • Pros: Conserves OS communication resources, such as socket handles and connection control blocks • Cons: harder to program, less efficient, & less deterministic • In nonmultiplexed connections each client uses a different connection to communicate with a peer service • Pros: Finer control of communication priorities & low synchronization overhead since additional locks aren't needed • Cons: use more OS resources, & therefore may not scale well in certain environments

Sync vs. Async Message Exchange • Asynchronous request/response protocols stream requests from client to server without waiting for responses synchronously • Multiple client requests can be transmitted before any responses arrive from a server • These protocols therefore often require a strategy for detecting lost or failed requests & resending them later • Synchronous request/response protocols are the simplest form to implement • Requests & responses are exchanged in a lock-step sequence. • Each request must receive a response synchronously before the next is sent

Message Passing vs. Shared Memory • Message passing exchanges data explicitly via the IPC mechanisms • Application developers generally define the protocol for exchanging the data, e.g.: • Format & content of the data • Number of possible participants in each exchange (e.g., point-to-point unicast), multicast, or broadcast) • How participants begin, conduct, & end a message-passing session • Shared memory allows multiple processes on the same or different hosts to access & exchange data as though it were local to the address space of each process • Applications using native OS shared memory mechanisms must define how to locate & map the shared memory region(s) & the data structures that are placed in shared memory

Sidebar: C++ Objects & Shared Memory Allocating a C++ Object in shared Memory void *obj_buf = … // Get a pointer to location in shared memory ABC *abc = new (obj_buf) ABC; // Use C++ placement new operator • General responsibilities using placement new operator • Pointer passed to placement new operator must point to a memory region that is big enough & is aligned properly for the object type being created • The placed object must be destroyed by explicitly calling the destructor • Pitfalls initializing C++ objects with virtual functions in shared memory • The shared memory region may reside at a different virtual memory location in each process that maps the shared memory • The C++ compiler/linker need not locate the vtable at the same address in different processes that use the shared memory • ACE wrapper façade classes that can be initialized in shared memory must therefore be concrete data types • i.e., classes with only non-virtual methods

Overview of the Socket API (1/2) Sockets are the most common network programming API available on operating system platforms • Originally developed in BSD Unix as a C language API to TCP/IP protocol suite • The Socket API has approximately two dozen functions classified in five categories • Socket is a handle created by the OS that associates it with an end point of a communication channel • Asocket can be bound to a local or remote address • In Unix, socket handles & I/O handles can be used interchangeably in most cases, but this is not the case for Windows

Overview of the Socket API (2/2) Local context management Connection establishment & termination Data transfer mechanisms Options management Network addressing

Taxonomy of Socket Dimensions The Socket API can be decomposed into the following dimensions: • Type of communication service • e.g., streams versus datagrams versus connected datagrams • Communication & connection role • e.g., clients often initiate connections actively, whereas servers often accept them passively • Communication domain • e.g., local host only versus local or remote host

Limitations with the Socket APIs (1/2) • Poorly structured, non-uniform, & non-portable • API is linear rather than hierarchical • i.e., the API is not structured according to the different phases of connection lifecycle management and the roles played by the participants • No consistency among the names • Non-portable & error-prone • Function names: read() & write() used for any I/O handle on Unix but Windows needs ReadFile() & WriteFile() • Function semantics: different behavior of same function on different OS e.g., accept () can take NULL client address parameter on Unix/Windows, but will crash on some operating systems, such as VxWorks • Socket handle representations: different platforms represent sockets differently e.g., Unix uses unsigned integers whereas Windows uses pointers • Header files: Different platforms use different names for header files for the socket API

Limitations with the Socket APIs (2/2) • Lack of type safety • I/O handles are not amenable to strong type checking at compile time • e.g., no type distinction between a socket used for passive listening & a socket used for data transfer • Steep learning curve due to complex semantics • Multiple protocol families & address families • Options for infrequently used features such as broadcasting, async I/O, non blocking I/O, urgent data delivery • Communication optimizations such as scatter-read & gather-write • Different communication and connection roles, such as active & passive connection establishment, & data transfer • Too many low-level details • Forgetting to use the network byte order before data transfer • Possibility of missing a function, such as listen() • Possibility of mismatch between protocol & address families • Forgetting to initialize underlying C structures e.g., sockaddr • Using a wrong socket for a given role

Example of Socket API Limitations (1/3) 1 #include <sys/types.h> 2 #include <sys/socket.h> 3 4 const int PORT_NUM = 10000; 5 6 int echo_server () 7 { 8 struct sockaddr_in addr; 9 int addr_len; 10 char buf[BUFSIZ]; 11 int n_handle; 12 // Create the local endpoint. Possible differences in header file names Forgot to initialize to sizeof (sockaddr_in) Use of non-portable handle type

Example of Socket API Limitations (2/3) 13 int s_handle = socket (PF_UNIX, SOCK_DGRAM, 0); 14 if (s_handle == -1) return -1; 15 16 // Set up address information where server listens. 17 addr.sin_family = AF_INET; 18 addr.sin_port = PORT_NUM; 19 addr.sin_addr.addr = INADDR_ANY; 20 21 if (bind (s_handle, (struct sockaddr *) &addr, 22 sizeof addr) == -1) 23 return -1; 24 Use of non-portable return value Protocol and address family mismatch Wrong byte order Unused structure members not zeroed out Missed call to listen()

Example of Socket API Limitations (3/3) 25 // Create a new communication endpoint. 26 if (n_handle = accept (s_handle, (struct sockaddr *) &addr, 27 &addr_len) != -1) { 28 int n; 29 while ((n = read (s_handle, buf, sizeof buf)) > 0) 30 write (n_handle, buf, n); 31 32 close (n_handle); 33 } 34 return 0; 35 } SOCK_DGRAM handle illegal here Reading from wrong handle No guarantee that “n” bytes will be written

ACE Socket Wrapper Façade Classes • ACE defines a set of C++ classes that address the limitations with the Socket API • Enhance type-safety • Ensure portability • Simplify common use cases • Building blocks for higher-level abstractions These classes are designed in accordance with the Wrapper Facade design pattern

The Wrapper Façade Pattern (1/2) Applications Solaris VxWorks Win2K Linux LynxOS • Context • Networked applications must manage a variety of OS services, including processes, threads, socket connections, virtual memory, & files • OS platforms provide low-level APIs written in C to access these services • Problem • The diversity of hardware & operating systems makes it hard to build portable & robust networked application software • Programming directly to low-level OS APIs is tedious, error-prone, & non-portable

The Wrapper Façade Pattern (2/2) calls API FunctionA() calls methods Application calls API FunctionB() calls API FunctionC() Wrapper Facade void method1(){ void methodN(){ functionA(); functionA(); data functionB(); } } method1() … methodN() : Application : Wrapper : APIFunctionA : APIFunctionB Facade method() functionA() functionB() • Solution • Apply the Wrapper Facade design pattern (P2) to avoid accessing low-level operating system APIs directly This pattern encapsulates data & functions provided by existing non-OO APIs within more concise, robust, portable, maintainable, & cohesive OO class interfaces

ACE Socket Wrapper Façades Taxonomy • The structure of the ACE Socket wrapper facades reflects the domain of networked IPC properties • The ACE Socket wrapper façade classes provide the following capabilities: • ACE_SOCK_* classes encapsulate Internet-domain Socket API functionality • ACE_LSOCK_* classes encapsulate UNIX-domain Socket API functionality • ACE also has wrapper facades for datagrams • e.g., unicast, multicast, broadcast

Roles in the ACE Socket Wrapper Facade • The active connection role (ACE_SOCK_Connector) is played by a peer application that initiates a connection to a remote peer • The passive connection role (ACE_SOCK_Acceptor) is played by a peer application that accepts a connection from a remote peer & • The communication role (ACE_SOCK_Stream) is played by both peer applications to exchange data after they are connected

ACE Socket Addressing Classes (1/2) • Motivation • Network addressing is a trouble spot in the Socket API • To minimize the complexity of these low-level details, ACE defines a hierarchy of classes that provide a uniform interface for all ACE network addressing objects

ACE Socket Addressing Classes (2/2) • Class Capabilities • The ACE_Addr class is the root of the ACE network addressing hierarchy • The ACE_INET_Addr class represents TCP/IP & UDP/IP addressing information • This class eliminates many subtle sources of accidental complexity

ACE I/O Handle Classes (1/2) • Motivation • The low-level C I/O handle data types are tedious & error-prone • Even the ACE_HANDLE typedef is still not properly object-oriented & typesafe int buggy_echo_server (u_short port_num) { sockaddr_in s_addr; int acceptor = socket (PF_UNIX, SOCK_DGRAM, 0); s_addr.sin_family = AF_INET; s_addr.sin_port = port_num; s_addr.sin_addr.s_addr = INADDR_ANY; bind (acceptor, (sockaddr *) &s_addr, sizeof s_addr); int handle = accept (acceptor, 0, 0); for (;;) { char buf[BUFSIZ]; ssize_t n = read (acceptor, buf, sizeof buf); if (n <= 0) break; write (handle, buf, n); } }

C++ Network Programming Mastering Complexity with ACE & Patterns