Disk and File Systems The Boot Process

1. Disk and File SystemsThe Boot Process

2. File Systems • One or more disks are attached to the machine • Each disk is divided into a number of partitions (partitioning is good to minimize the lose of data) • In unix, each partition is mounted somewhere on the file system, at a directory point except the swap partition. • There always will be the root partition on the system (the mount point is / ). • Mount point of the file systems provides transparency of the file systems for the user ( allows mounting files systems from other systems)

3. File System • The mounted file systems can be viewed by df command. • For example, df -k on our Linux server will produce an output similar to this. • Swap has traditionally been to a partition,but some systems allow swapping to file (security,performance issue need to considered)

4. File Systems • Maximum file size is limited by the size of a partition. • Files (at least user files) cannot span partitions (or file systems). • Partitions cannot be expanded dynamically. • Each partition usually has one type of file systems (ufs, msdos, Advfs, etc)

5. Different Type of File Systems • Mainly to improve performance or provide a different mix of compromises. • The issues in file system performance may include • read vs write speeds • handling of large vs small files • robustness to system crashes • memory requirements for buffers • slacks space

6. Different Type of File Systems • ufs - unix file systems • Advfs - Advanced file systems mainly designed for Digital/Compaq True unix • ufs (ffs) - Berkeley Fast File system • nfs - Network File Systems, originally from sun Microsystems. Used to mount file systems from other systems through network. • lfs - Log Srtructure File System. Also known as Vertas file system, Journalled file System (on IBM aix), Advanced File System (OSF)

7. Ufs - Unix Fast File System • The most common unix file system currently in use. • Disk is divided up into cylinder groups. • Each cylinder group contains some house keeping information such as a copy of the super block, inodes, etc. • House keeping information is kept at varying offsets within the cylinder group to avoid total destruction in the event of a head crash. • Unit of transfer is a block/ • The block allocation policy attempts to maximizelocality without significantly affecting performance.

8. Startup & shutdown • When the machine is turned on, it searches its own hardware for a bootable file. • Usually Linux system is started by LILO (the LInux Loader), a program that can boot multiple operating systems. • LILO is placed on the master boot record. However for NT or OS/2 which require complete control of the master boot record, the LILO has to be placed somewhere else.

9. The Boot process • Load the kernel into the memory • Initialize the kernel • Probe the hardware • Creation of system processes • Execute initialization scripts • go-multi-user

10. Boot Media and Firmware • Many systems can boot from media other than disk, e.g tape, network, CD-ROM. You nee to know the device name given by Linux. • Some firmware can be configured to ask for password on boot up (Do this only if you cannot guarantee physical security of the machine - for example, what happens when there is power failure - system may not come up automatically) • Any file systems which require a password before mounting?

11. Loading and Initializing the Kernel • Boot sector loaded from he hard disk by firmware monitor. • Waits for input and parameters (if any). Timeouts and goes with the defaults (LILO provides 10 secs to type-in) • Loads in partitions table 9ofthe disk) • Looks for an active partition (generally first) • Loads kernel that was specified or default (vmlinuz). • Jumps to predefined location which starts the kernel. • Initializing the kernel may involve • determining physical memory • may relocate itself depending on amount of memory, size of kernel, etc • dynamically calculates file buffers and other system structures • Reports on remaining memory • Comprehensive console output for debugging purposes

12. Hardware Probing • Kernel has a list of device drivers. Each is called to probe for a particular device • Device driver reports success or failure. If success Linux notes this and calls an initialization routine in the driver. • Console output indicates which devices have been detected. • Devices added after the boot process not detected until the next boot process. • Many device drivers are configured into default kernel. Can save memory and time customizing kernel and removing the unnecessary device drivers. • Some kernels can automatically configure themselves, though this assumes that the hardware auto-detection process is flawless. • On Linux this information is displayed on the console However you can usedmesg [ -c] [ -n level ] [ -s buffer]to display the boot probe message.

13. Linux Boot Diagnostics

14. Linux Boot Diagnostics

15. Linux Boot Diagnostics • To get the boot message from your last system startup, typedmesg | moredmesg [ -c] [ -n level ] [ -s buffer]-c - Clear the contents of the ring buffer after printing-sbufsize - specify the size of the buffer used to query the kernel ring buffer. Default size is 8196.-nlevel - specify the level at which messages are displayed to the console. Note that messages are sent to proc/kmsg regardless of level. Typically, this is set to 1, which filters all but panic messages.

16. Linux Boot Process • Regardless of the boot method, the first thing that happens is that the kernel is loaded. • The kernel initializes its own data data structures and loads its device drivers. • Then the kernel starts the init program which,in turn, forks off clones to create the various user processed. • Linux uses the concept of runlevels to define which processes has to be started This system variable takes a value between 0 to 6.Runlevel Meaning 0 system halted 1 administrative runlevel 2,3,4,5 user definable run levels 6 reboot runlevel • When the system transitions between runlevels, it consults the /etc/inittab file to see what processes to start or stop.

17. /etc/inittab file contents

18. /etc/inittab file • Field 1: Identifier (unique 14 characters) • Field 2: The run levels in which this line is to be processed (0 = halted, 2=mulituer with no network, etc) • filed 3: Action respawn= initstarts if process dies, wait=init runs thisproces and waits for its completion, once=same as wait but do not wait for completion, boot=run this command first time inittab is read, bootwait=same as boot but wait for completion, powerfail=execute this command when SIGPWR is received, powerokwait=same as powerfail but power is restored and hence terminate the shutdown, initdefault=specifies default run evel when init starts, sysinit = run this program before requested run level • Field 4: Any valid shell command (max 1024 bytes) • note where the rc0, rc2 ad rc3 scripts are run and respawn command on the last line.

19. Linux Boot Process • A great deal of boot time system configuration is accomplished via scripts. • The /etc/rc.d/rc{runlevel}.d directories contain script associated with the various run levels. • These scripts are executed by init in order of their appearance in the output of the ls command upon entering that run level.

20. Shutdown • When shutdown command is invoked, the system sends messages notifying users of the impending shutdown. • Usually it is a good practice to give them a bit of warning lest that come back from their coffee break (or something similar) and find a day’s work is destroyed or lost. • Next the executing processes are sent a signal and they terminate with varying degree of grace. • The subsystems are shutdown, and users who did not take the hint are kicked out (by sure kill signal) and any process that did not respond to the signal are killed. • Any file system updates are written by the sync command and finally init process takes the system to the new run level (usually to 0 for halt). • A shutdown can be aborted by getting hold of the process-id and killing it manually - Why you want to do this?

21. Startup/Shutdown commands • Command Descriptiondmesg display bootup messagehalt halt the systemreboot reboot the systempoweroff power the system offlilo install the lilo boot loaderrdev display system startup configuration inforunlevel show the current system runlevelshutdown bring the system downswapon enable the paging hardwareswapoff disable the paging h/wsync write buffered memory out to diskinit start system processestelinit move the system to a new runlevel

22. Related files for startup/shutdown • /var/log/wtmp Login records file/fastboot This file’s existence constitutes a request by the system to skip fsck at reboot time/etc/inittab defines what processes are to be started at what run level/etc/shutdown.allow Uses specified in his file are permitted to invoke he shutdown command./etc/rc Multiuser initialization file/etc/fstab contains mount information for file systems/etc/hd? Disk device/etc/sd?? SCSI disk device/etc/initscript If exist,used by init to start new processes/etc/ioctl.save Specifies console’s ioctlstates/etc/lilo.conf specifies LILO configuration options

23. Exercise • Objectives of this practical exercise is to • Find out how to get information on the file systems • Rebooting and shutting down the machines. • You may have access to your local as well as the Linux server machine (use the root account on the local machine and the supplied user name on the Linix server) • Use df command to find the file systems that are mounted and interpret the output. • Use df –i command to check the number of free I-nodes (why we need to know this?)

24. Exercise • Look at the file /etc/fstab and interpret its contents with the output of df command. • Use the disklabel –r hda and understand its output. This command reads information about the disk regarding the partitions. For most unix systems, it is rarely necessary to have to actually modify the disk label, though knowing how to get it can be useful (especially when you add a new disk). How to look at other disk labels?

25. Exercise • When a file system is repaired it will often have files truncated. The truncated files are unfortunately referred by I-node number rather than file name.The command pstat –v | moreshows the currently opened files and directories on the machine by displaying the vnode table which is kept in the kernel.The column headed FILEID corresponds to the inode number. Use find asfind / -inum 1234to find the names of the truncated files.

26. Exercise • Learn to mount and umount DOS formatted floppy drive (before mounting a floppy drive make sure that the diskette is inserted into the drive – if not what happens?). You need a mount point for mounting the floppy. Create a mount point directory called /floppy at the root file system. • Issue commands to shutdown the machine gracefully (by giving some time) or to bring the machine to a halt. • If you have installed multiple OS, see how you can modify the lilo so that the default boot OS can be changed?

27. Exercise • Assuming /floppy, /cdrom are the mount points for the floppy and cdrom, the following commands will mount themmount –t msdos /dec/fdo /floppymount –t iso9660 /dev/cdrom /cdrom • mount command can be used to mount partitions on local hard drives. This is very useful for accessing and sharing files between Windows and Linix. Windows 9x partitions are usually FAT 16 or FAT 32. Linux will recognize FAT file systems and they are classified vfat when mounting. The commandmount –t vfat /dev/hda1 /winwill mount the first partition of hda (usaully Windows 9x) at the /win mount point.

28. /etc/fstab • The Linux file system table, or /etc/fstab is a file that can be used to predefine mount points and other variables. Every time the mount command is issued, the /etc/fstab is read for this purpose. Using a text editor (vi), and while logged in as root, edit the /etc/fstab. Assume that you have the following line in this file/dev/fdo /floppy msdos noauto,rwthen to mount the floppy simply typemount /floppyThis will work properly because the mount conditions for each type of file system have been set in the /etc/fstab file.

29. Super user and Scheduling (root and Daemons ]

30. Process Creation • Associated with each process is a user and group ID that determine the privileges of that process in accessing resources. • Typically the user and group ID of a process are those of the user that requested the process be run, but this is not always the case. • Some processes are created with higher privileges in order to accomplish their particular job. These are called set uid and set gid processes.

31. Process Creation Computer ls Run ls ls process • Stage 1 - Our user (id=srini, group id = users) types ls and hits return. This what the id command says when srini runs it (the computer is called beauty)beauty> iduid=1000(srini) gid=1000(users) groups=1000 (users) • Stage 2 - The ls command is loaded from disk. On disk it is owned by user bin, and is part of the bin group.beauty> ls -l /bin/ls-rwxr-xr-x 1 root root 44828 Mar 31 00:13 bin/ls • Stage 3. The ls command is executed in memory with user id = srini and group id= usersbeauty>ps -l | grep ls

32. Set Group ID Process creation Computer man man process Run man • The /usr/bin/man command is loaded from disk and these are the attributes of that program-rwxr-sr-x 1 root man 35196 Mar 1 23:59 manNotice there is an ‘s’ in the group execute position. The file is owned by the user root and belongs to group man. • Our user srini (id = srini, group =users) types man ps • The man command is executed in memory with user id = srini, but the group id = man!! Why?

33. Set User ID Process creation • The /bin/crontab command is loaded from disk and these are the attributes of that program-rwsr-xr-x 1 root bin 35196 Mar 1 23:59 crontabNotice there is an ‘s’ in the owner execute position. The file is owned by the user root and belongs to group man. • Our user srini (id = srini, group =users) types crontab srini • The crontabcommand is executed in memory with user id = root !! and group id = users. Why? • crontab program need to create files in the /var/cron/tabs directory and this directory is owned by root

34. Setuid and Setgid • File modes, indicated by lowercase ‘s’ in place of the x (execute permission), e.g (setuid) –rwsr-xr-x, (setgid) rwxr-sr x • Setuid – set user id on execution of program. Some programs are setuid root,which means that upon execution the programs run as if they are run by root (with the associated privileges of the root) • Setgid – set group id on execution of the program. • Real and effective ID’s (user and group) determines whay resources a process has access to. • Setuif programs which are not properly written can present a security problem.

35. Superuser (root) Privileges • User id = 0 • Has access rights to all files and resources in the system. • Many tasks can only be performed by root. • su command and root password allows ordinary user to become root, provided they are in the root group.

36. Controlling access to root • Control via /etc/group • Those users not in group 0 cannot become root via the su command. • Controlling the terminals root logins are allowed on. • /etc/ttys file lists those terminals that the system knows about. Additionally a secure password can be specified next to the terminal entry if the root is allowed to logon at that terminal. • Generally password access to root does not exist in single user mode. If there is concern for physical security of the machine, then it is to have a password for the root in single user mode. To boot linux in single user typelinux –s at boot time

37. Single User • Only one user, the person at the console. • No password protection generally, so physical security is the only defense!! • Not many processes running, generally only init and the the shell you are using. • Generally the prompt is # • Limited set of tools, varies depending on system, but usually /bin, /sbin and /etc are the available directories (usually the / partition, hence make sure when you install the system, the required tools are in the root partition)

38. Getting into Single-user mode • Set boot flag either the fireware monitor (like sun boxes) while on Linux –PCs, invoke boot –s. • Once operating system is running use shutdown command. • During the boot process if a problem occurs then the system system may enter single user mode. Typical problems may include • Not all devices in /etc/fstab can be mounted (typically this occurs when a file system is corrupted, run fsck to correct the errors) • Init cannot find the initialization scripts or /etc/inittab file • The initialization script hangs

39. Getting out of Single user mode • <ctrl>d will exit and take you into multi-user mode provided there are no problems with the boot process. • Otherwise issue reboot or halt command.

40. What use is single-user mode? • Only one user on the system, less activity, less damage if a crash occurs. • Can set date/time without causing problems with running programs (like cron). This is not common now because of the on-board hardware clock itself. • It is the minimal mode that the system can boot if other file systems are corrupt. • Allows to umount the /usr and other normally busy file systems. • Man pages man not be immediately available (note system administrators!!)

41. Single user mode • Normally the / file system is available, which means the executable in /bin and /sbin are available (unless they are installed on other file systems – which may not be a good idea). • Only limited set of tools, most that deals with the files systems are available. Even vi may not be available!! • Possible to access to other file systems only by mounting.

42. Some tools in single user mode • ed – line based editor, good for changing configuration filesbadset – find and record which sectors are baddisklabel – read/write/modify disklabels (this allows to change the disk partitions)dump – dump a file system (generally to a tape or disk)fsck – check and repair a file systemifconfig – configure a network interfacemount/umount – mount (unmount) file systemsnewfs – make a new file system on a partitionreboot/halt

43. fsk – file system checker • Checks to see of file system was un-mounted cleanly. • Uses redundant copies of information to check the integrity of disk data. • Able to recreate housekeeping information if disk is damaged. • Checks that are made are • Superblocks • number of free blocks • individual inode corruption • iode links • number of allocated blocks • Directories • Most often fsck is run automatically by the initialization scripts at the boot time.

44. Demons • Programs that run in the background. • Outputs (if any) usually to log files. • Most demons are started by the initialization scripts when the machine is booted(you might have specified when you installed the system). • Generally perform periodic and monitoring activities. • Can be seen by ps –a command

45. The normal demons that runs on Linix

46. The start up script where demons are started at boot time

47. at and its related commands • These commands are used to tell the system to do something at (or until) a later time. • Proper sue of these commands is the key to an efficiently run system. These include, but not limited to • System security check (password file, file access,…) • Cleanup temporary files (rm /tmp) • Updating system databases (updatedb) • Backups and archiving (tar, shar)

48. at and its related commands • at add a job to the scheduling queueatq Display the contents of the scheduling queueatrm Remove a job from the scheduling queue batch Execute queue only when system resources permitcrontab Maintain schedule of jobs to be executedsleep Wait some number of secondsusleep Wait some number of microseconds

49. at and its related files • The related files for the previous set of commands are/var/spool/at List of at commands to be spooled/var/spool/at/spool Spooled at commands/proc/loadavg Contains system load average/var/run/utmp Database of users currently logged in/etc/at.allow Users permitted to use at command/etc/at.deny Users prohibited from using at/etc/cron.allow Users permitted to use cron/etc/cron.deny Users prohibited from using cron

50. at and its related commands • The syntax of the commands:at [-V] [-q queue] [-f file] [-mldbv] TIMEat –c job [job, …]atq [-v] [-1 queue] [-v]atrm [-V] job [job…] • Specifying TIME: 3 ways to specify the time: absolute, relative or verbal. • Absolute: MM/DD/YY or DD.MM:YY or MMDDYY (with or without HH:MM) • Relative: <some absolute time> + delay where delay is <some number>[mts|hours|days|weeks] • Verbal: noon, midnigh, teatime, AM, PM, tomorrow, +