1 / 35

Nachos Instructional OS: Part 3

Nachos Instructional OS: Part 3. CS 170, Tao Yang, Fall 2011. Topics. File system implementation. Assignment 3 Virtual memory File system extension. Nachos File System. Two versions (check openfile.h and filesys.h): A ``stub'' version is simply a front-end to the Linux.

Télécharger la présentation

Nachos Instructional OS: Part 3

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Nachos Instructional OS: Part 3 CS 170, Tao Yang, Fall 2011

  2. Topics • File system implementation. • Assignment 3 • Virtual memory • File system extension

  3. Nachos File System • Two versions (check openfile.h and filesys.h): • A ``stub'' version is simply a front-end to the Linux. • A Nachos version implemented on top of a raw disk. • Function layers (from top to down): • FileSystem object • Directory support. • I-node support (FileHeader) • Free block map • OpenFile objects (many) • Created by every call to open/access a file in a directory. • SynchDisk object • Created by synchDisk = new SynchDisk("DISK") in system.cc • The Disk object. • Created by disk = new Disk(name,..) in SynchDisk::SynchDisk()

  4. File System Layers in Nachos FreeMap Bitmap for 32x32 blocks Nachos kernel thead Directory: Manage a directory of file names FileSystem: Directory management file create/open/delete FileHeader: i-node OpenFile: access individual file with read/write/seek. Operate on bytes. SynchDisk: synchronous access to a disk with concurrent threads. Operate on sectors. Disk: low-level interface that initiates asynchronous IO to physical disk sectors Base Operating System (Linux for our class)

  5. Operations of File System Object • FileSystem(bool format). • called once at ``boot'' time. • For the stub version, just use Unix file system • Otherwise create a directory/control info. • Sector 0 is fileheader (i-node) for FreeMap. • Sector 1 is i-node for directory table. • Create(char *name, int initialSize). • Create a Nachos disk file. ``initialSize'' is ignored in the stub version; Unix doesn't require it. • OpenFile *Open(char *name). • Assume both read and write access. • in the stub version, Open simply opens the Unix file. • Remove(char *name) • List()

  6. Directory Table stored in DirectoryFile • Sector 1 is the i-node (file header). • File content is 10 table entries. • DirectoryFile disk space is allocated in FileSystem() during initialization … Bool inUse --- is this entry used? Int sector --- location of file header (i-node) Filename -- 9 characters+ 1 for ‘\n’ …

  7. directory.cc • Limit • FileNameMaxLen 9 // 9 characters at most for a file name • NumDirEntries 10 // at most 10 files • Directory Object attributes • tableSize -- # of directory entries which is 10 now. • table = in-memory directory entries, each of which is for one Nachos file. Content is copied from and copies back to a disk DirectoryFile. • Operations • Directory (int size) -- Create an in-memory directory of the desired size. • FetchFrom (OpenFile *file) -- Fetch disk DirectoryFile that contains directory entries, and copy to in-memory directory table. • WriteBack(OpenFile *file) -- Write back in-memory directory content to disk DirectoryFile • called every time when a file is added or removed to directory. • FindIndex (Char *name) --Search a file string name in a directory • Add (Char *name, int sector) – Add file name into a directory with disk sector no that contains the file header (i-node). • Remove(char *name) – remove a file.

  8. File header (i-node) … int numBytes; // file size int numSectors; //sectors used • One filer header per file, fitting one sector (128 bytes) • Operations • Allocate(BitMap *bitMap, int fileSize); Initialize a file header • FetchFrom (int SectorNo) -- Fetch Fileheader sector from disk. • WriteBack(int SectorNo) -- Write back in-memory file header content to disk • ByteToSector(int offset) –Convert a file offset in bytes to a sector no Data sector int dataSectors[NumDirect] Data sector pointer 0 Data sector Data sector pointer 1 … Data sector Data sector pointer 29

  9. OpenFile Object • OpenFile(int sector) • sector – the location on disk of the file header. • Create an OpenFile object which contains file control info, loaded from the disk file header. • Seek(int position) • Read/write in the current position • Read(char *buffer, int numBytes) • Write(char *buffer, int numBytes) • Read/write in a specific position • ReadAt(char *buffer, int numBytes, int FilePosition). • Identify specific sectors in the file and fetch them, copy to buffer. • WriteAt(char *buffer, int numBytes, int FilePosition) • May read partially written sectors first, and then write out updated sectors. • Length(). Return the file size.

  10. SynchDisk Object: Synchronously access a disk • SynchDisk () • Called only once to create a physical disk object. Set the disk interrput hander as RequstDone(). • Create lock/semaphore for synchronization. • Notice that physical disk is an asynchronous device (disk requests return immediately, and an interrupt happens later on). • ReadSector(int sectorNumber, char* buffer) • Send a read request for a sector • Wait IO complete (wait for interrupt using a semaphone) • WriteSector(int sectorNumber, char* data) • Send a sector write request. Wait for interrupt. • RequestDone() • Disk interrupt handler. Wake up any thread waiting for the disk request to finish.

  11. Disk Object • Simulates the behavior of a disk I/O device. • The disk has only a single platter, with multiple tracks (32). • Each track contains the same number of sectors (32). • Each sector is 128 bytes • Disk size = 32x32*128 =128K • Asynchronous read/write with an option of disk cache • Supported operations: • Disk(char *name, VoidFunctionPtr callWhenDone, int callArg) • Create a disk: just create a Unix file to simulate. • Setup a handling function (RequestDone in SynchDisk.cc) when a read/write request is done.

  12. Operations in Disk device • ReadRequest(int sectorNumber, char *buffer) • Read from the specific location of the “physical disk”. • ticks = ComputeLatency(sectorNumber, FALSE); • interrupt->Schedule(DiskDone, (int) this, ticks, DiskInt); • WriteRequest(int sectorNumber, char *data) • Write data to specific location of the “physical disk”. • ticks = ComputeLatency(sectorNumber, TRUE); • interrupt->Schedule(DiskDone, (int) this, ticks, DiskInt); • ComputeLatency(int newSector, bool writing) • Latency= seek time + rotation time

  13. Assignment 3 Workload • vm directory for part 1 is empty • Need to come up a design and code • Test C programs provided are sample code. • Amount of work • Part 1. ~400 lines of code in vm ( few changes in userprog) • Part 2. ~250 or less in filesys (few in userprog) • Part 1 and 2 can be done in parallel

  14. Project 3 part 1: Virtual Memory • Work on vm subdirectory mainly • And addrspace.cc/.h and exception.cc in userprog • Create/manage a backing store (a file called SWAP using the OpenFile class). • Implement a page fault handler with dirty bit handling and a page replacement policy (LRU or second chance) • Test under various conditions: • One process with an address space larger than physical memory. • Concurrent processes with combined address space larger than physical memory.

  15. Project 3 part 2: File system • Work on filesys subdirectoy mainly. • And change system call handling code in userprog if needed (to use this extended file system). • Extend the size of each file from 4K to 100K • A few single indirect pointers in i-node. • The size of a file can grow.

  16. Report to be submitted •  HW3_WRITEUP • Summarize what is completed, what is not. • For VM, describe the design of VM. • For file system extension, • Describe design options and their trade-offs (for i-node and for handling file size growth. • List/explain main components and code modification in implementing your design • it will be graded: Do you write down the design, list advantages/disadvantages of options? Explain the reason of your choice? • Summarize the test effort (what is passed, the purpose of each test)

  17. Emphasis of Project 3 •  Play tradeoffs • Understand the design options, especially for addressing required issue in file system extension. • Deadline-driven approach • Choose something simpler and easier to do first. • Still not enough time? • Complete with minimal efforts to pass the tests. • Hidden tests are very similar to public test cases. No complex test cases. • You are good as long as your code can run virtual memory>physical, can support growable file up to 100K. • Interpret the requirement towards your advantage if vague. If not specified, do something easier.

  18. HW 3 Implementation Notes

  19. Part 1: Getting Started • Read machine/translate.cc: • In Machine:Translate() for virtual address translation, PageFaultException is detected when the desired page is not in memory. • In Machine:ReadMem, Translate() is called for translating the desired virtual memory address and machine->RaiseException() is called with PageFaultException error.

  20. What is next • Read mipssim.cc • Machine->ReadMem() is called in executing each instruction. • If PageFaultException is detected, the exception handler should load the desired page. • The hardware will try again. • Need to expand exception.cc to handle PageFaultException. • Once handled, return to user mode and restart the instruction caused the fault

  21. User Instruction Execution Re-execute if Exception is raised Machine:Run () OneInstruction () ReadMem () WriteMem () Machine:Translate() Page writing? Set dirty bit Cannot find this page? Raise PageFaultException ExceptionHandler() Deal with PageFaultException

  22. Files to be modified for Part 1 • New files in directory vm • Virtual memory manager • Swap space manager • Directory userprog (extension to HW 2) • exception.cc • Extension to handle PageFaultException • Addrspace.cc/.h • Prepare code/data for SWAP backstore. • Virtual address translation -> paging if needed

  23. PageFaultException handling • For a virtual address which is not in memory, • Find free space to host the desired page in memory. • Load the desired page from SWAP to memory. • If no free space available, execute a replacement policy to find a victim page. • Swap out the victim page. • Swap in the desired page. • Adjust the page table.

  24. Suggested Class in vm • Write a Swap Manager to facilitate free sector allocation in Swap File. • Initialize SWAPSIZE (512) as the total free sector available. • Allocate a sector. • Free a sector.

  25. Suggested Class in vm • Virtual memory manager • load a page containing the virtual address (copy from the SWAP file to an allocated memory page). • Find a victim page • Swap out a selected page to SWAP

  26. Modify AddSpace.cc • In translating a virtual user address for kernel, if it is not in memory, bring data from SWAP. • When allocating a new address space, • Allocate a proper number of sectors from SWAP for this process. • Copy binary data to the allocated SWAP sectors. • Initial page number for this process can be 0.

  27. Synchronization issues • Two system calls may be processed concurrently and the synchronization is needed. • Any time a process sleeps in the kernel, another process can run • Two processes try to initiate I/O on the same page at the same time • May need a lock (or at least a ``busy'' flag) for each physical memory page, and possibly also for each virtual memory page.

  28. Part 2: Steps for Nachos file system extension • Understand how the Nachos file system operates and how Nachos implements files. • Modify the file system calls from HW 2 to use Nachos file system • Modify Nachos file system so that you can create larger (100K) • Allow the Write system call to grow the size of a file.

  29. Files to be modified in Part 2 • Directory filesys • filehdr.cc/.h Support single indirect pointers in i-node and can expand the file length with more sectors allocated. • openfile.cc Expand file length/allocate sectors when writing data at the end or beyond. • directory.cc Add more table entries when the current table is full and a new file is added. • Directory userprog • exception.cc • Make sure that file system calls use extended Nachos file system.

  30. Comparison: Unix file i-node (4K bytes per block)

  31. Step 3: Extend maximum file size • Modify the FileHeader i-node to support 100K size. • Include 5 data sector pointers, and 25 sectors for single indirect mapping. • At most map 5 + 25*32=805 blocks. Maximum file length = 805*128=103040bytes. • Another option: every entry points to an i-node. • Suggest to add a class called IndirectPointerBlock which takes 1 sector (32 entries) with following operations. • FetchFrom (int sector). WriteBack(int sector). • AddSector (int sector) – add a new sector. • Deallocate(BitMap *freeMap) – deallocate all sectors. • ByteToSector (int offset). – convert an offset in bytes to a sector.

  32. Step 4: File size can grow • Modify the current implementation so that a write beyond the end of the file extends the size of the file. • Another option is: have a fixed size initially to allocate the i-node space initially. But donot allocate data blocks until they are needed. • Gracefully handle the case of the disk being full or maximum size is reached - the user process should not crash. Instead an error should be returned.

  33. Step 4 • Modify WriteAt() method in openfile.cc : • Case 1: Write request is within the bound of the file. Handle like the existing code • Case 2: Write request overwrites some portion of existing file and goes beyond by N bytes. • Case 3: No overwrite to the current data, but goes beyond the end of the file. Here. • To extend a file dynamically, • Extend i-node/data dynamically: include a routine in FileHeader to extend a file by N bytes. • Or pre-allocate the i-nodes, allocate data sectors when needed.

  34. Testing issues • Clean out all the .c files in the code/test directory. There might be name conflicts from previous projects • Get sample test programs in part1 and part2 of the ~cs170/nachos-projtest/HW3 directory. • Test part 1. Test as you would for project 2. You can run the ./nachos in the vm/ • Test part 2. It can be convenient to create another test directory (e.g. test under test2 directory or fielsys/test)  • Notice file names cannot exceed 9 characters.

  35. Testing issues in Part 2 • May need to load user binaries/data into Nachos file system during execution • Example: csil$ pwd cs170/nachos/code/filesys csil$ ./nachos -f -cp ../test2/Prog1 Prog1 -x Prog1 • Another example with two binary programs csil$ ./nachos -f -cp ../test2/Prog2 Prog2 -cp ../test2/Prog3/Prog3 -x Prog2

More Related