Case Study – The Linux OS Part II

1. Case Study – The Linux OSPart II Mostly taken from: Silberschatz – Chapter 20 Nutt – Chapter 20

2. Virtual Memory • The VM system maintains the address space visible to each process: It creates pages of virtual memory on demand, and manages the loading of those pages from disk or their swapping back out to disk as required • The VM manager maintains two separate views of a process’s address space: • A logical view describing instructions concerning the layout of the address space • The address space consists of a set of nonoverlapping regions, each representing a continuous, page-aligned subset of the address space • A physical view of each address space which is stored in the hardware page tables for the process Linux Operating System - Part -II - Slide 2 Fakhar Lodhi

3. Virtual Memory (Cont.) • On executing a new program, the process is given a new, completely empty virtual-address space • The program-loading routines populate the address space • Creating a new process with fork involves creating a complete copy of the existing process’s virtual address space • The kernel copies the parent process’s VMA descriptors, then creates a new set of page tables for the child • The parent’s page tables are copied directly into the child’s, with the reference count of each page covered being incremented • After the fork, the parent and child share the same physical pages of memory in their address spaces • Copy-on-write mechanism Linux Operating System - Part -II - Slide 3 Fakhar Lodhi

4. Virtual Memory (Cont.) • The VM paging system relocates pages of memory from physical memory out to disk when the memory is needed for something else • The VM paging system can be divided into two sections: • The pageout-policy algorithm decides which pages to write out to disk, and when • The paging mechanism actually carries out the transfer, and pages data back into physical memory as needed Linux Operating System - Part -II - Slide 4 Fakhar Lodhi

5. Virtual Memory (Cont.) • The Linux kernel reserves a constant, architecture-dependent region of the virtual address space of every process for its own internal use • This kernel virtual-memory area contains two regions: • A static area that contains page table references to every available physical page of memory in the system, so that there is a simple translation from physical to virtual addresses when running kernel code • The reminder of the reserved section is not reserved for any specific purpose; its page-table entries can be modified to point to any other areas of memory Linux Operating System - Part -II - Slide 5 Fakhar Lodhi

6. Executing and Loading User Programs • Linux maintains a table of functions for loading programs; it gives each function the opportunity to try loading the given file when an exec system call is made • The registration of multiple loader routines allows Linux to support both the Executable and Linking Format (ELF) and a.out binary formats • An ELF-format binary file consists of a header followed by several page-aligned sections • The ELF loader works by reading the header and mapping the sections of the file into separate regions of virtual memory • Initially, binary-file pages are mapped into virtual memory • Only when a program tries to access a given page will a page fault result in that page being loaded into physical memory Linux Operating System - Part -II - Slide 6 Fakhar Lodhi

7. Mapping of Programs into Memory Top of user mode Points to the current extent Variable size region Fixed size region Write protected Linux Operating System - Part -II - Slide 7 Fakhar Lodhi

8. Static and Dynamic Linking • A program whose necessary library functions are embedded directly in the program’s executable binary file is statically linked to its libraries • The main disadvantage of static linkage is that every program generated must contain copies of exactly the same common system library functions • Dynamic linking is more efficient in terms of both physical memory and disk-space usage because it loads the system libraries into memory only once Linux Operating System - Part -II - Slide 8 Fakhar Lodhi

9. Linux File Manager • Defines a single internal view of files that can be used to read/write files onto a storage device using a variety of file managers • A file is viewed as a named byte stream that can be saved in the secondary storage • Supports directory and file operations • Implements asynchronous read/write operations Write Read File Manager I/O Buffers Device Driver Storage Device Linux Operating System - Part -II - Slide 9 Fakhar Lodhi

10. Linux File Manager – VFS • Virtual File System (VFS) • Implements FS independent operations • VFS switch – provides the API to user space programs and establishes an internal interface used by different file system translators • The VFS inode contains details about the file. Linux File Manager System Call Interface VFS Switch MS-DOS MINIX Ext2 /proc Linux Operating System - Part -II - Slide 10 Fakhar Lodhi

11. File Systems • To the user, Linux’s file system appears as a hierarchical directory tree obeying UNIX semantics • Internally, the kernel hides implementation details and manages the multiple different file systems via an abstraction layer, that is, the virtual file system (VFS) • The Linux VFS is designed around object-oriented principles and is composed of two components: • A set of definitions that define what a file object is allowed to look like • The inode-object and the file-object structures represent individual files • the file systemobject represents an entire file system • A layer of software to manipulate those objects Linux Operating System - Part -II - Slide 11 Fakhar Lodhi

12. The Linux Ext2 File System • Ext2 performs its allocations in smaller units • The default block size on ext2fs is 1Kb, although 2Kb and 4Kb blocks are also supported • Ext2 uses allocation policies designed to place logically adjacent blocks of a file into physically adjacent blocks on disk, so that it can submit an I/O request for several disk blocks as a single operation Linux Operating System - Part -II - Slide 12 Fakhar Lodhi

13. The Linux Ext2 File System • Two parts • Partition into multiple block groups • Each group belongs to a single cylinder of the physical disk • May be Different sizes of groups • ext2 selects the block group for the file • tries to keep the related information in the same group block • for data block it tries to choose the same block group in which the file’s inode has been allocated • for inodes it selects the same group as the file’s parent directory • directory files are not kept together to spread out disk load • within the block groups, it tries to keep the allocation physically contiguous • maintains a bitmap of all free blocks in a block group Linux Operating System - Part -II - Slide 13 Fakhar Lodhi

14. The Linux Ext2 File System • when allocating the first block for a new file, it starts searching for a free block • for extending the file it continues the search from the block most recently used • the search is performed in two stages • first it searches for an entire free byte in the bitmap – aims to allocate disk blocks in chunks (at least 8 where possible). • if it fails then it searches for any free bit • once a free block has been identified, the search is extended backwards until an allocated block is encountered • when a free byte is found in the bitmap. this backward extension prevents leaving a hole between the most recently allocated block in the previous non-zero byte and the zero byte found. • extends the allocation forward for up to eight blocks and preallocates these extra blocks to the file to reduce fragmentation and save disk allocation time. • preallocated blocks are returned to free space when the file is closed. Linux Operating System - Part -II - Slide 14 Fakhar Lodhi

15. Ext2 Block-Allocation Policies if we can find any free blocks sufficiently near the start of the search then we allocate them no matter how fragmented Linux Operating System - Part -II - Slide 15 Fakhar Lodhi

16. The Linux Ext2 File System (cont.) • Ext2 internals divided into group • Super block, Group descriptor, Block descriptor, inode bitmap, inode table, data blocks • Different block size length • 1KB, 2KB, 4KB, 8KB • Root has 5% of space reserved • Enable filesystem recovery Linux Operating System - Part -II - Slide 16 Fakhar Lodhi

17. The Linux Ext2 File System - inode • The inode contains important information regarding the file • Kernel keeps the inodes of open files in memory • A structure that contains file’s description: • Type • Access rights • Owners • Timestamps • Size • 15 Pointers to data blocks • First twelve points to data block. • Thirteenth is called indirect pointer • Fourteenth is called doubly indirect pointer • Fifteenth is called triple indirect pointer Linux Operating System - Part -II - Slide 17 Fakhar Lodhi

18. inode structure File Info pointers to data blocks indirect pointer IND double indirect pointer DIND triple indirect pointer TIND Linux Operating System - Part -II - Slide 18 Fakhar Lodhi

19. Directories • A directory is just a special type of file • These are structured in a tree hierarchy • Each can contain both files and directories • Special user-functions for directory access • Each dentry contains filename + inode-no • Kernel searches the direrctory tree • translates a pathname to an inode-number Linux Operating System - Part -II - Slide 19 Fakhar Lodhi

20. Directory diagram Inode Table Directory name1 i1 name2 i2 name3 i3 i4 name4 Linux Operating System - Part -II - Slide 20 Fakhar Lodhi

21. Links • Multiple names can point to same inode • The inode keeps track of how many links • If a file gets deleted, the inode’s link-count gets decremented by the kernel • File is deallocated if link-count reaches 0 • This type of linkage is called a ‘hard’ link • Hard links may exist only within a single FS • Hard links cannot point to directories (cycles) Linux Operating System - Part -II - Slide 21 Fakhar Lodhi

22. The Linux /proc File System • The /proc file system does not store data, rather, its contents are computed on demand according to user file I/O requests • proc must implement a directory structure, and the file contents within; it must then define a unique and persistent inode number for each directory and files it contains • It uses this inode number to identify just what operation is required when a user tries to read from a particular file inode or perform a lookup in a particular directory inode • When data is read from one of these files, proc collects the appropriate information, formats it into text form and places it into the requesting process’s read buffer Linux Operating System - Part -II - Slide 22 Fakhar Lodhi

23. The Linux /proc File System • The /proc file system is a direct reflection of the system kept in memory and represented in a hierarchal manner. • The effort of the /proc file system is to provide an easy way to view kernel and information about currently running processes. • As a result, some commands (ps for example) read /proc directly to get information about the state of the system. • The premise behind /proc is to provide such information in a readable manner instead of having to invoke difficult to understand system calls. Linux Operating System - Part -II - Slide 23 Fakhar Lodhi

24. Input and Output • Linux splits all devices into three classes: • block devices allow random access to completely independent, fixed size blocks of data • character devices include most other devices; they don’t need to support the functionality of regular files • network devices are interfaced via the kernel’s networking subsystem • User cannot directly transfer data to network devices • All device drivers appear as normal files • The Linux device-oriented file system accesses disk storage through two caches: • Data is cached in the page cache, which is unified with the virtual memory system • Metadata is cached in the buffer cache, a separate cache indexed by the physical disk block Linux Operating System - Part -II - Slide 24 Fakhar Lodhi

25. Device-Driver Block Structure Block Devices • Provide the main interface to all disk devices in a system • The block buffer cache • serves two main purposes: • it acts as a pool of buffers for active I/O • it serves as a cache for completed I/O • The request manager manages the reading and writing of buffer contents to and from a block device driver Linux Operating System - Part -II - Slide 25 Fakhar Lodhi

26. Device-Driver Block Structure Character Devices • A device driver which does not offer random access to fixed blocks of data • A character device driver must register a set of functions which implement the driver’s various file I/O operations • The kernel performs almost no preprocessing of a file read or write request to a character device, but simply passes on the request to the device • The main exception to this rule is the special subset of character device drivers which implement terminal devices, for which the kernel maintains a standard interface Linux Operating System - Part -II - Slide 26 Fakhar Lodhi

27. Device-Driver Block Structure Network Devices • Networking is a key area of functionality for Linux. • It supports the standard Internet protocols for UNIX to UNIX communications • It also implements protocols native to nonUNIX operating systems, in particular, protocols used on PC networks, such as Appletalk and IPX Linux Operating System - Part -II - Slide 27 Fakhar Lodhi

28. Device-Driver Block Structure Network Devices • Internally, networking in the Linux kernel is implemented by three layers of software: • The socket interface • Protocol drivers • Network device drivers Linux Operating System - Part -II - Slide 28 Fakhar Lodhi

29. Device-Driver Block Structure Network Devices • The most important set of protocols in the Linux networking system is the internet protocol suite • It implements routing between different hosts anywhere on the network • On top of the routing protocol are built the UDP, TCP and ICMP protocols Linux Operating System - Part -II - Slide 29 Fakhar Lodhi

30. Interprocess Communication • Linux informs processes that an event has occurred via signals • There is a limited number of signals, and they cannot carry information: Only the fact that a signal occurred is available to a process Linux Operating System - Part -II - Slide 30 Fakhar Lodhi

31. Passing Data Between Processes • The pipe mechanism allows a child process to inherit a communication channel to its parent, data written to one end of the pipe can be read a the other • Each pipe has a pair of wait queues to synchronize readers and writers Linux Operating System - Part -II - Slide 31 Fakhar Lodhi

32. Shared Memory Object • Shared memory offers an extremely fast way of communicating; any data written by one process to a shared memory region can be read immediately by any other process that has mapped that region into its address space Linux Operating System - Part -II - Slide 32 Fakhar Lodhi

33. Security – Pluggable Authentication Module (PAM) • PAM provides a way to develop programs that are independent of authentication scheme. • By using PAM, multiple authentication technologies, such as RSA, DCE, Kerberos, smart card and S/Key, can be added without changing any of the login services, thereby preserving existing system environments. • Which authentication module is to be attached is dependent upon the local system setup and is at the discretion of the local system administrator. • PAM is based on a shared library that can be used by any system component that needs to authenticate users Linux Operating System - Part -II - Slide 33 Fakhar Lodhi

34. Security – Pluggable Authentication Module (PAM) • Access control under UNIX systems, including Linux, is performed through the use of unique numeric identifiers (uid and gid) • Access control is performed by assigning objects a protections mask, which specifies which access modes—read, write, or execute—are to be granted to processes with owner, group, or world access Linux Operating System - Part -II - Slide 34 Fakhar Lodhi

35. End of Linux Overview