Linux Interprocess Communication

1. Linux Interprocess Communication Valter Borges, Cristos Screpetis, and Andrew Miner Computer Science & Engineering Department The University of Connecticut 191 Auditorium Road, Box U-155 Storrs, CT 06269-3155 Process Creation Linux

2. Fork() • Fork() is a function that allows the programmer to create a new process. • “Parent” is the process that executes the fork() and creates a new process. • “Child” is the newly created process. When initiated it contains a copy of the parent’s data.

3. Fork() • When a process terminates it is not completely destroyed. The parent needs to execute a wait() and wait for the process to finish. • If a parent process is terminated with one or more child processes still running, the child’s new parent will become the init process. • When the child process terminates, without the original parent waiting, it becomes de-functioning. Such processes are eventually killed by the init process.

4. Fork() Sample Fork() Code ... switch(pid=fork()) { case -1: perror("fork"); /* something went wrong */ exit(1); /* parent exits */ case 0: printf(" CHILD: This is the child process!\n"); ... default: printf("PARENT: This is the parent process!\n"); printf("PARENT: My … }

5. Linux Interprocess Communication Kernel IPC Linux

6. Signals Signal is a message that can be passed between processes, and it can be used to interrupt, kill, report errors etc. A signal is raised (generated) by a process and delivered to another process to process it. The process’s signal handler, which is associated with that specific signal, is then executed. The following is an example of a signal handler, that overrides the default, SIGINT handler (the handler for the interrupt signal)

7. Signals SIGINT Example int main(void) { … /*Handler setup */ if (signal(SIGINT, sigint_handler) == SIG_ERR) { perror("signal"); exit(1); } …. } /* this is the handler */ void sigint_handler(int sig) { /*Code to be executed when the SIGINT is received */ ... }

8. Signals Signals are usually generated by the system, and delivered to the currently running processes. Here is list of the most common signals that are raised.

9. Wait Queues Wait Queues hold an ordered list of processes that wait to be executed. When the wait queue is processed, the processes that is scheduled to run removes herself from the queue and starts executing. The wait queues are very useful in synchronizing access to system resources. The processes entered in the queue can be either interruptible or un-interruptible.

10. File Locks • File locks is a mechanism that is used to coordinate file accessing on a system. The use of file locks prevents errors generated by processes that try to access the same file at the same time. • There are two types of of file locks: • A read file lock the locks the file for reading. • A write file lock that locks the file for writing.

11. File Locks • There are two types of locking: • Mandatory, which prevents access to the file by a process other than the one executing the lock for both read and write locks. • Advisory, which may allow another process (other than the one executing the read or write lock) to access the file at the same time. • The advisory type usually allows more than one processes to execute read locks on the same file, at the same time, but it prevents more than one write locks on the same file by two or more processes at the same time.

12. File Locks Sample File Locking Code … fd = open("filename", O_WRONLY); /* get the file descriptor */ fcntl(fd, F_SETLKW, &fl); /* set the lock, waiting if necessary */ . . . fl.l_type = F_UNLCK; /* tell it to unlock the region */ fcntl(fd, F_SETLK, &fl); /* set the region to unlocked */ …

13. Memory Mapped Files Memory mapped files are used as a communication system between processes. The can provide faster, and more efficient access to a file, or a section of a file, by mapping it to a section of the memory. . Instead of opening and closing the file, the processes can now access it directly by making references to a memory location. When the file is mapped to memory, the process(s) that need to access that file, or a section of it, are provided with a pointer to the memory location where the file is mapped.

14. Memory Mapped Files The following is simple example of mapping a file to memory: fd = open("mapdemofile", O_RDWR); pagesize = getpagesize(); data = mmap((caddr_t)0, pagesize, PROT_READ, MAP_SHARED, fd, pagesize); The file is first opened, and then mapped to memory. The arguments passed are the memory address where the file is mapped, the file’s length, protection and sharing flags, the file descriptor, and the size of a virtual memory page.

15. Linux Interprocess Communication BSD IPC Linux

16. Networking Sockets • Sockets provide processes with data communication. • A client-server interface is established between two or more processes, with one of the processes being the serverand one or more client processes. • A server process listens to the socket for requests coming from the client processes, and then processes their requests. • A client process on the other hand, using the socket’s interface, sends requests to the server process, according to its needs.

17. Networking Sockets Here is a brief description of the initialization of a socket by both server and client processes. Server Process The server processes first gets a socket descriptor by making a call to socket(). It then bind()s that with an address in the UNIX domain, and begins listening to the socket for a request from a client. Client Process The client process on the other hand, only needs to call socket(), and then establish a connection to the socket by calling connect(), and using the server’s remote address. It can then send requests to the server by using the socket’s interface.

18. Networking Sockets Sample Code Server: s = socket(AF_UNIX, SOCK_STREAM, 0); ... bind(s, (struct sockaddr *)&local, len); listen(s, 5); ... Client: … s = socket(AF_UNIX, SOCK_STREAM, 0); connect(s, (struct sockaddr *)&remote, len); ...

26. Example for FIFO's: #include <stdio.h> #include <stdlib.h> #include <sys/stat.h> #include <unistd.h> #include <linux/stat.h> #define FIFO_FILE "MYFIFO" int main(void) { FILE *fp; char readbuf[80]; /* Create the FIFO if it does not exist */ umask(0); mknod(FIFO_FILE, S_IFIFO|0666, 0); while(1) { fp = fopen(FIFO_FILE, "r"); fgets(readbuf, 80, fp); printf("Received string: %s\n", readbuf); fclose(fp); } return(0); }

27. Linux Interprocess Communication System V IPC Linux

28. Linux Interprocess Communication • AT&T System V • 1979 Unix Version 7 (Bell Labs) is released for general use. • 3 major flavors of the Unix system emerge: • BSD (Berkley System Distribution) • Xenix • AT&T’s System V. • 1983 AT&T releases their version of Unix called System V. AT&T

29. Linux Interprocess Communication • Linux adopts System V • Linux provides support for the three types of System V interprocess communication mechanisms. • The three types are: • Message Queues • Semaphores • Shared Memory System V

30. Linux Interprocess Communication • Linux IPC Goals • Linux System V IPC mechanisms as well as other IPC mechanisms have the following goals: • Provide a means for concurrent processes to share resources. • Allow concurrent processes to synchronize and exchange data with one another. I P C

31. File IPC FIFO PIPES System V IPC Msg. Queues Shared Memory BSD/ Net IPC Semaphores Domain Sockets Kernel IPC Wait Queues Signals IPC Subsystem System Call Interface

32. IPC Process Scheduler Memory Manager File System IPC Subsystem Dependencies

33. IPC Subsystem Dependencies File System and IPC -IPC subsystem depends on the file system for sockets, which use file descriptors, and once open are assigned to an inode. -File system depends on IPC for synchronization and for abstractions such as file locks, and memory-mapped files. Memory Manager and IPC -The Memory manager depends on IPC for the swapping of shared memory. -IPC depends on the Memory manager for buffer allocation and implementation of shared memory. Process Scheduler -IPC mechanisms rely on timers implemented by the scheduler. -Process Scheduler relies on signals from the Kernel IPC system.

34. System V IPC Objects • Common Properties • -All created in the kernel, and each have associated access permissions. • -Processes may access these resources only by passing a unique reference identifier to the kernel via system calls. • -Share common authentication methods. • -All include an “ipc_perm” structure which contains: • Owner and Creator processes user and group identifiers • Access mode for the object (owner, group, and other). • Object Key

35. Kernel ipc_perm structure -The ipc_perm structure is defined ipc.h as follows: struct ipc_perm { key_t key; ushort uid; /*owner */ ushort gid; ushort cuid; /*creator*/ ushort cgid; ushort mode /* octal access mode*/ ushort seq; /* sequence number-keep track of max number of objects that can reside in a system*/ };

36. IPC Identifiers • -Each IPC object has a unique IPC identifier associated with it. • -The identifier is used by the kernel to uniquely identify the IPC object. • -Uniqueness of an identifier is relevant to the type of object. • -An IPC object can consist of: • single message queue • semaphore set • shared memory segment

37. IPC Keys -A key must be used in order to obtain a unique identifier. -The key is associated and used as a way of locating the System V IPC object’s resource identifier. -The object’s reference indentifier is used as an index into a table of resources. Two sets of key’s supported: Public-Any process in the system, subject to rights checking can find an object’s reference identifier. Private-Only kernel has access to the object. -System V IPC objects can only be referenced by their reference identifier.

38. Creating a Key -Key generation is up to the discretion of the application programmer. -Key can be hardcoded since key_t is just a long, but this has the disadvantage of the key already being in use. -Normally function ftok() can alleviate this problem. generation. Prototype: key_t ftok(char *pathname, char proj) Returns: key if successful, -1 otherwise. Ex: key_t mykey; mykey= ftok(“/tmp/myapp”, ‘a’);

39. Creating a Key (Continued) -ftok combines the inode number and minor device number from the pathname with the char project identifier. This does not guarantee uniqueness but it is possible to check for collisions and retry key generation. -Once key value is obtained it is used in a subsequent IPC system calls to generate the unique identifier. -The unique identifier will be used to gain access to IPC objects.

40. Useful Commands The ipcs Command -Used to obtain the status of all System V IPC objects ex: ipcs -q: Show only message queues ipcs -s: Show only semaphores ipcs -m: Show only shared memory sample output: ---------------------Shared Memory Segments-------------- shmid owner perms bytes nattch status --------------------Semaphore Arrays------------------------ semid owner perms nsems status --------------------Message Queues--------------------------- msqid owner perms used-bytes messages 0 root 660 5 1

41. Useful Commands (Continued) The ipcrm Command -Used to remove an IPC object from the kernel. -Use the ipcs command to obtain the IPC ID ex: ipcrm <msg | sem | shm> <IPC ID>

42. Message Queues Basic Concepts -Message queues allow one or more processes to write messages, which will be read by one or more reading processes. -Message queues work almost like a named pipe, with some additional functionality. -Most of the time messages are taken out in the order they are put in, but it’s possible to cut the line. -Message queues can be created or connected to, by connecting to a message queue two or more processes can share information. -A message Queue doesn’t go away until destroyed, even if all the processes quit. Use ipcs and ipcrm to manage your Queues.

43. Message Queues Internals -Linux maintains an internal linked list of message queues within the kernel’s addressing space, it is called the msgque vector. -Each elemement of the msgque vector points to a msqid_ds data structure that fully describes the message queue. -Each time a message queue is created a new msqid_ds data structure is allocated from memory and inserted into the linked list. -Within the msqid_ds there exists an ipc_perm structure, pointers to the first and last messages, modification times, pointers to the kernel’s wait queue, size and number of messages, and pid of last to write.

44. Message Queues (A Closer Look) -A message buffer is a structure that is defined in msg.h but can be redefined by the programmer for the specific application but you must keep the first field a long, which should always be positive. There is a limit to the maximum size of a given message and it’s defined in msg.h to be 4056 this max size for the whole structure. In msg.h struct msgbuf { long mtype; /* type of message */ char mtext[1]; /*message text */ My Redefinition struct my_msgbuf { long mtype; char name[60]; int age; };

45. Message Queues (A Closer Look) Kernel msg structure -Each message in the queue is stored in a msg structure by the kernel. Define in msg.h struct msg { struct msg *msg_next; /* next message on queue */ long msg_type; /*Assigned from user’s msgbuf char *msg_spot; /*message text address */ short msg_ts; /* message text size */ };

46. Message Queues (A Closer Look) Kernel msqid_ds structure -Each msg queue in the system has an instance of the msqid_ds data structure associated with it. Define in msg.h struct msqid_ds { struct ipc_perm msg_perm; /* instance of the ipc_perm structure */ struct msg *msg_first; /* first message in the queue */ struct msg *msg_last /* last message in the queue */ time_t msg_stime; /* time last msg was sent */ time_t msg_rtime; /* time last msg was retrieved*/ time_t msg_ctime; /* time last change made to the queue*/ struct wait_queue *wwait, *rwait; /*pointers to kernel’s wait queue, used when full*/ ushort msg_cbytes, msg_qnum; /*number of messages and number of bytes on queue*/ ushort msg_qbytes; /* maximum number of bytes on the queue*/ ushort msg_lspid, msg_lrpid; /* PID of processes who sent and retrieved last message };

47. msqid_ds msg msg messg messg Message Queues (A Closer Look) System V IPC Message Queues Kernel’s Link List *msg_first ………... msg_qnum

48. Creating a Message Queue To create a new message queue or access an existing queue System Call: msgget() Prototype: int msgget( key_t key, int msgflg); Returns: Either a msg queue id for a newly created queue or the id of an existing queue. -1 on error: errno = EACCESS(permission denied) EEXISTS (Queue exists, cannot create) EIDRM (Queue is marked for deletion) ENOENT (Queue does not exist) ENOMEM(Not enough memory to create queue) ENOSPC (Maximum queue limit exceeded) msgflg: IPC_CREAT- Create the queue if it doesn’t already exist in the kernel. IPC_EXCL - When used with IPC_CREAT, fail if queue already exists. This field must be set to IPC_CREAT and OR’d with the permissions for the queue. If IPC_EXCL is OR’d to IPC_CREAT it will guarantee that no existing queue is opened for access.

49. Sending to the Queue To send a message to the queue System Call: msgsnd() Prototype: int msgsnd( int msqid, struct msgbuf *msgp, int msgsz, int msgflg); Returns: 0 on success -1 on error: errno = EAGAIN (queue is full, and IPC_NOWAIT was asserted) EACCES (permission denied, no write permission) EFAULT (msgp address isn’t accessible - invalid) EIDRM (The message queue has been removed) EINTR (Received a signal while waiting to write) EINVAL(Invalid msg queue id or message size, or nonposit msg type) ENOMEM (Not enough memory to copy msg buffer) msqid - Queue identifier obtained using msgget() msgp - pointer to our message buffer msgsz - length o message in bytes minus the size of message type msgflg - Set to 0 is ignored, or set to IPC_NOWAIT. IPC_NOWAIT - If msg queue full then msg not written and control returned, if not used then calling process will block until msg is written.

50. Retrieving from the Queue To retrieve a message from the queue System Call: msgrcv() Prototype: int msgrcv (int msqid, struct msgbuf *msgp, int msgsz, long mtype, int msgflg) Returns: Number of bytes copied into message buffer -1 on error: errno = E2BIG (Message length is greater than msgsz) EACCES (No read permission) EFAULT (Address pointed to by msgp is invalid) EIDRM (The message queue has been removed) EINTR (Received a signal while waiting to write) EINVAL (Invalid msg queue id or message size, or nonposit msg type) ENOMSG (IPC_NOWAIT asserted, and no such msg exists in the queue) msqid - Queue identifier obtained using msgget() msgp - pointer to our message buffer msgsz - length o message in bytes minus the size of message type msgtyp - 0 to retrieve then next msg, Positive to get msg with mtype equal, Negative to get first msg whose mtype is less than or equal to the absolute value of msgtyp. msgflg - Set to 0 is ignored, or set to IPC_NOWAIT. IPC_NOWAIT - Non-blocking call with possible returns: ENOMSG, EIDRM, EINTR