Introduction to the Microsoft exFAT File System - PowerPoint PPT Presentation

htcia 2014 international conf hyatt lost pines resort austin texas tuesday august 26 2014 8 00am n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Introduction to the Microsoft exFAT File System PowerPoint Presentation
Download Presentation
Introduction to the Microsoft exFAT File System

play fullscreen
1 / 121
Introduction to the Microsoft exFAT File System
162 Views
Download Presentation
Download Presentation

Introduction to the Microsoft exFAT File System

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. HTCIA 2014 International Conf‎Hyatt Lost Pines Resort, Austin TexasTuesday August 26, 2014 8:00am Introduction to the Microsoft exFAT File System Robert Shullich CPP, CISSP, CRISC, GSEC, GCFA, CEH, CHFI, CCFP-US HTCIA 2014 Conf - Aug 26, 2014

  2. Agenda • About me, the paper and the presentation • The need for a new generation of FAT • Digital Forensics Relevance • Exponents and Standards • exFAT Overview • Linux Development • Memory Cards & Flash Memory • exFAT File System Internals • Closing HTCIA 2014 Conf - Aug 26, 2014

  3. About me, the paper and the presentation About Me About the Presentation About the SANS Paper A Gold Standard Another Paper Reference Disclaimer exfat HTCIA 2014 Conf - Aug 26, 2014

  4. About Me I have been in the IT field for 40+ Years, and in InfoSec for over 20 Years I carry many IT and InfoSec certifications This research was originally for a class term project towards my D4CS MS degree I then expanded that term paper into a practical paper for my SANS “Gold” GCFA certification Links to the SANS paper and my blog are provided at the end of this presentation HTCIA 2014 Conf - Aug 26, 2014

  5. About the Presentation What I call the exFAT Road Show • The New York Forensics Computer Show 4/20/2010 • Techno Security and Digital Investigations 6/7/2010 • SANS What Works in Forensics and IR Summit 7/8/2010 • HTCIA International Training Conference & Expo 9/20/2010 • The New York Forensics Computer Show 4/19/2011 • http://techchannel.att.com/play-video.cfm/2011/8/16/Conference-TV-Computer-Forensics-Show:-Introduction-to-exFAT • NYC4SEC 6/11/2014 • HTCIA International Training Conference & Expo 8/26/2014 HTCIA 2014 Conf - Aug 26, 2014

  6. About the SANS Paper • Consider it “exFAT – the missing manual” • Very little published about exFAT today • Two current forensics books mention exFAT: • Wiley - Mastering Windows Network Forensics and Investigation • Sybex - EnCase Computer Forensics - The Official EnCE: EnCase Certified Examiner • “For those seeking an in-depth understanding of the exFAT file system, you should read the SANS paper entitled “Reverse Engineering the Microsoft Extended FAT File System (exFAT) by Robert Shullich” HTCIA 2014 Conf - Aug 26, 2014

  7. A Gold Standard • 2005 Book considered the authority on different file systems • The book’s Author developed the open-source TSK forensics tools (The Sleuth Kit) & Autopsy • This year adding exFAT to TSK HTCIA 2014 Conf - Aug 26, 2014

  8. Another Paper Reference HTCIA 2014 Conf - Aug 26, 2014

  9. Disclaimer • The released specification and implementation is Release 1.00 of exFAT • The specification mentions additional features that were not implemented yet, but may at a future time/ Some of these are Windows CE holdovers • Both may be presented today • Some directory entries will be skipped • Focus is Microsoft Desktop/Server implementation • Will talk about Flash/Solid State, but high level • For exFAT, tried to stay with the patent terminology HTCIA 2014 Conf - Aug 26, 2014

  10. The need for a new generation Legacy FAT Why do we need a new file system? Why do we need Faster I/O and Higher Capacity? Hi-definition movie recording MPEG-4. H.264 exfat HTCIA 2014 Conf - Aug 26, 2014

  11. Legacy FAT • FAT 8 • 1977 Bill Gates and Marc McDonald • Floppy based • FAT 12 • 1980 • FAT 16 • 1984 with release of PC/AT & MS DOS 3 • FAT 16B • 1987 Compaq DOS 3.31 • FAT 16X • 1995 PC DOS 7.0/Win 95 – LBA Addressing • FAT 32 • 1996 Windows 95 OSR2, 98, ME, MS DOS 7.1 – CHS Addressing • FAT 32X • LBA Addressing HTCIA 2014 Conf - Aug 26, 2014

  12. Why do we need a new file system? • Current Limits Exhausted (Ran Out of Bits!) • Larger volumes (>2TB) (Scale to Larger Capacity) • Larger files sizes (>4GB) • Faster I/O • (UHS-I: 104 MB/s - UHS-II: 312MB/s) • Removable Media • Flash/Solid State Media • Flexibility • Extensibility (Difficult to add new features) • NTFS Features without the overhead • Easier to implement FS in firmware HTCIA 2014 Conf - Aug 26, 2014

  13. Why do we need Faster I/O and Higher Capacity? http://www.cnet.com/news/what-is-4k-uhd-next-generation-resolution-explained/ HTCIA 2014 Conf - Aug 26, 2014

  14. Hi-def movie recording MPEG-4. H.264 HTCIA 2014 Conf - Aug 26, 2014

  15. Digital Forensics Relevance Relevance to Forensics Study What happens when you have exFAT formatted media and no exFAT support? Forensics Challenges in 2009 Forensics Challenges Today exfat HTCIA 2014 Conf - Aug 26, 2014

  16. Relevance to Forensics Study • Digital Evidence Extraction • Finding the evidence • Including the hiding places • Validation • Completeness • Daubert Expert Testimony • Need to know and understand file org • Establish Credibility • New Media (SD Cards) will drive exFAT adoption, and the potential for CP investigations. • Larger Media Capacity also driving exFAT adoption HTCIA 2014 Conf - Aug 26, 2014

  17. Trust but Verify HTCIA 2014 Conf - Aug 26, 2014

  18. What happens when you have exFAT formatted media and no exFAT support? HTCIA 2014 Conf - Aug 26, 2014

  19. Forensics Challenges 2009 • In 2009, in regards to exFAT: • No tools (RAW) • No documentation or Training • No expertise • Evidence backlog HTCIA 2014 Conf - Aug 26, 2014

  20. Forensics Challenges Today • Today • exFAT Misunderstood • Linux OS Support • Tuxera drivers may help (Embedded) • FUSE and No-FUSE hacks • Most Distributions – No native support • Mac OS Support (Nov 2010) OS/X 10.6.5+ • Implementation Deviations, No Standards • Open Source Tools • Commercial Tools • Encase (6.14.3 Dec 2009) • Encase (6.18.0.59) NIST Test March 2014 • FTK (3.2 Oct 2010) • FTK (3.3) NIST Test April 2014 • Cross Vendor Compatibility HTCIA 2014 Conf - Aug 26, 2014

  21. NIST Computer Forensics Tool Testing • Cyber Fetch • AAFS-2013 Conference 02/21/2013 • Deleted File Recovery Tool • Testing Results • One Summary Item: Support for ExFAT, ext3 & ext4 is sometimes lacking. HTCIA 2014 Conf - Aug 26, 2014

  22. Test Results for Deleted File Recovery and Active File Listing • 17 Basic Tests • March 12, 2014 – Encase 6.18.0.59 • MAC differed by 9 hours • April 3, 2014 – FTK 3.3.0.33124 • MAC differed by 4 hours • The exFAT partition and HFS+ created on OS/X 10.6 • exFAT: ctime meta-data replaced with the time of file deletion [I was unable to recreate] • Vendor Tool or Apple Implementation? • Who Validates the Test? HTCIA 2014 Conf - Aug 26, 2014

  23. Who Validates the Validator? Superman: Easy, Miss, I've got you Lois Lane: You...you've got me, who's got you? HTCIA 2014 Conf - Aug 26, 2014

  24. Exponents and Standards Base 2 or 10? Exponents International System of Units (SI) Table IEC 60027-2 Reference Standards Endian Microsoft Math More Math – exFAT WinCE exfat HTCIA 2014 Conf - Aug 26, 2014

  25. Base 2 or 10? HTCIA 2014 Conf - Aug 26, 2014

  26. Exponents 102 = 10 times 10 = 100 103 = 10 times 10 times 10 = 1000 (1K) 22 = 2 times 2 = 4 29 = 2*2*2*2*2*2*2*2*2 = 512 210 = 2*2*2*2*2*2*2*2*2*2 = 1024 (1K) 212 = 2*2*2*2*2*2*2*2*2*2*2*2 = 4096 HTCIA 2014 Conf - Aug 26, 2014

  27. International System of Units (SI) Table • File System in powers of 2 • Device characteristics in power of 10 HTCIA 2014 Conf - Aug 26, 2014

  28. IEC 60027-2 http://physics.nist.gov/cuu/Units/binary.html HTCIA 2014 Conf - Aug 26, 2014

  29. How far off are we? http://cnx.org/content/m13081/1.1/ HTCIA 2014 Conf - Aug 26, 2014

  30. Reference Standards • Bits are numbered right to left • 76543210 • Decimal Offsets (zero based) • Little-Endian numbers • Unsigned numbers • Sectors vs. Clusters • Strings are 16 bit Unicode • Strings not Terminated HTCIA 2014 Conf - Aug 26, 2014

  31. Endian • Numbering order may vary based on processor type, is determined by the order the data bytes are read from the register. • A 32 bit number is read as 4 8-bit bytes • If I have the number 0x11 22 33 44 • Big-Endian will store it as: • 0x 11 22 33 44 • Little-Endian will store it as: • 0x 44 33 22 11 HTCIA 2014 Conf - Aug 26, 2014

  32. Microsoft Math KB184006 Limitations of FAT32 File System The maximum possible number of clusters on a volume using the FAT32 file system is 268,435,445. With a maximum of 32 KB per cluster with space for the file allocation table (FAT), this equates to a maximum disk size of approximately 8 terabytes (TB). 512B Sectors in a 32 KB cluster = 64 228 (268,435,445) * 26 (64) * 29 (512) = 243 = 8,796,093,022,208 Size of FAT32 FS specified in BPB as sectors (32 bit number) HTCIA 2014 Conf - Aug 26, 2014

  33. More Math, exFAT KB955704 Description of the exFAT file system driver update package • Support for volumes that are larger than 32 GB, the theoretical maximum volume size for FAT32 in Windows XP • The theoretical maximum volume size is 64 ZB. • The recommended maximum volume size is 512 TB. • Support for files that are larger than 4 GB, the theoretical maximum file size for FAT32 in Windows XP • The theoretical maximum file size is 64 ZB. • The recommended maximum file size is 512 TB. HTCIA 2014 Conf - Aug 26, 2014

  34. WinCE HTCIA 2014 Conf - Aug 26, 2014

  35. Overview Features of exFAT 1.00 4K (4096) Sector Size Supported Cluster Sizes Features of exFAT 1.00 (cont’d) Future Features of exFAT MBR Partition Limitations Advantages of exFAT Disadvantages of exFAT OS Support for exFAT Key Dates for exFAT exfat HTCIA 2014 Conf - Aug 26, 2014

  36. Features of exFAT 1.00 • Maximum Volume Size (Increased Capacity) • Architectural ≈ 128 PiB (232-11 * 225) • Implementation = 512 TiB • Sector sizes from 512 [SF] to 4096 bytes [AF] • Clusters sizes to 32MiB (225) • Subdirectories to 256MiB (Root not restricted) • Maximum files on volume ≈ 232 • Maximum File Size 16 EiB-1 • Built for speed, less overhead than NTFS • Catches up with some NTFS features • Template-based metadata structures • On-disk storage of file Valid Data Length (VDL) • Speeds up storage allocation processes HTCIA 2014 Conf - Aug 26, 2014

  37. 4K (4096) Sector Size HTCIA 2014 Conf - Aug 26, 2014

  38. Supported Cluster Sizes HTCIA 2014 Conf - Aug 26, 2014

  39. Features of exFAT 1.00 (cont’d) • OEM Parameters Sector for device dependent parameters • 12 sector VBR, support of larger boot program • Up to 2,796,202 files per sub-subdirectory • File Names max to 255 Characters • 16-Bit Unicode File Names and Volume Labels • Optimized for Flash Memory • Device Boundary Alignment • No FAT32 minimum cluster (65,525) restriction • No 8.3 file name support (only LFN) • UTC Timestamp Support • Vista/Server 2008 SP2+, XP/Server 2003 with KB • Native in Windows 7, 8, 8.1, Server 2008 R2, 2012 HTCIA 2014 Conf - Aug 26, 2014

  40. Future Features of exFAT • TexFAT (To be released later) • Exists in Windows CE • Transaction Safe exFAT • ACL (To be released later) • Exists in Windows CE • Compression & Encryption Support? • Not announced, but would be easy to add HTCIA 2014 Conf - Aug 26, 2014

  41. MBR Partition Limitations • Microsoft File Systems are limited when stored in a MBR partition • A partition is defined by a Master Boot Record • A MBR uses a 4 byte value for number of sectors • LBA as 32 bit # times 512 Sector limits to 2TiB • To get the maximum volume size, exFAT cannot be created within a MBR partition, Need GPT GUID Partition, or Super floppy Mode • ExFAT on GPT works on Mac HTCIA 2014 Conf - Aug 26, 2014

  42. Advantages of exFAT • Large volume, file and directory sizes • Handle growing capacities in media, increasing capacity to >32 GB. • > 1000 files in a single directory. • Speeds up storage allocation processes. • Breaks file size 4 GB barrier. • Supports interoperability with future desktop OSs. • Provides an extensible format. • Large cluster sizes • Metadata integrity with checksums HTCIA 2014 Conf - Aug 26, 2014

  43. Disadvantages of exFAT • Not all Windows CE features implemented • No direct conversion to or from other FS • Cannot use CONVERT command to NTFS • No Floppy Support • Mostly a Microsoft Desktop and Server World • No Support for Older MS systems (Pre-XP) • Support for other devices, surfacing • No Information Sector “Hint” • Like all FAT – Finding Stuff is via brute force HTCIA 2014 Conf - Aug 26, 2014

  44. OS Support for exFAT • Windows XP & Server 2003 • KB955704 (requires SP2 or SP3) • Vista & Server 2008 SP1 • Vista & Server 2008 SP2 • (Adds UTC timestamp support) • Windows 7/Server 2008 R2 and later: • RTM • Mac OS/X 10.6.5 and later HTCIA 2014 Conf - Aug 26, 2014

  45. Key Dates for exFAT • September 2006 – Windows CE 6.0 • March 2008 – Windows Vista Service Pack 1 • January 2009 – Announcement at CES of SDXC specification • January 2009 – Windows XP Drivers Available • May 2009 – Windows Vista Service Pack 2 • August 2009 – Tuxera Signs File System IP Agreement with Microsoft • March 2009 – Pretec Releases first SDXC Cards • December 2009 – Microsoft (re)announces exFAT license program for third-parties • December 2009 – SDXC laptops due soon • December 2009 – Diskinternals releases exFAT recovery utility • December 2009 – Encase support HTCIA 2014 Conf - Aug 26, 2014

  46. More Key Dates for exFAT • December 2009 Sony, Canon & Sanyo License • January 2010 Funai License (LCD TV) • February 2010 Panasonic License • February 2010 Panasonic 64/48GB SDXC • February 2010 Sony Memory Stick XC • February 2010 SanDisk Ultra SDXC 64GB Card 3.0 Spec $350 • April 26, 2010 DCF Version 2.0 (Edition 2010) • June 1st 2010 Tuxera Releases Linux & Android exFAT drivers • June 3rd 2010 Kingston Releases Class 10 SDXC 64GB Card 60 MB/s read, 35 MB/s write. • October 11th, 2010 FTK 3.2 with exFAT support announced HTCIA 2014 Conf - Aug 26, 2014

  47. More Key Dates • Mar 16th 2011 Lexar Releases SDXC 128GB • May 3rd, 2011 e.solutions (Volkswagen) • Aug 8, 2012 Sharp for Android Smart Phones • Sep 18, 2012 RIM (Blackberry) Smartphones • Nov 7, 2012 Sharp, Sigma, NextoDi, Black Magic and Atomos Global • Jan 16, 2013 BMW • April 30, 2014 PS4 V1.7 update – hidden new feature: exFAT HTCIA 2014 Conf - Aug 26, 2014

  48. Linux Development FUSE Project Samsung (No-FUSE) exfat HTCIA 2014 Conf - Aug 26, 2014

  49. Linux Development • Open Source community developing FUSE • FUSE – File System in User Space • Samsung accidently leaks native exFAT implementation, dubbed NO-FUSE • Samsung source code on GitHUB with GPLLicense • Still legal issues because of patent protection HTCIA 2014 Conf - Aug 26, 2014

  50. FUSE Project HTCIA 2014 Conf - Aug 26, 2014