Saturday, May 18, 2013

FAT32


Overview FAT32
For computer storage, there will have 2 type of address that are physical address and logical address. In OS level, you only can see the logical address where the address will use by FAT. The physical address is storage device actually memory address, this physical address is hide from the OS level.
Fdisk format (low level format) will format from physical address 0 and create Master boot record and  logical address pointer. Where this sector will be hidden sector in OS level.
The OS format (high level format) will create the FAT32 format in logical address 0.
FAT32 is A file system that required part of the Operating system (windows). It will contain storage volume, storage property, directory, folder, file information and data. It contain information that use to manage folder and file, and file's data pointer. The FAT32 uses a 32-bit file allocation table entry. However, FAT32 reserves the first bits of a FAT32 file allocation table entry, which means FAT32 has a theoretical maximum of 2^28 clusters. The removable disks are always formatted as FAT.
FAT file system volume is composed of 4 basic regions, which are laid out in this order on the volume:-
0 - Reserved region
1 - FAT region
2 - Root directory region (doesn't exist on FAT32 volumes)
3 - File and Directory Data Region

Organization of a FAT Volume

Contents Boot
sector
File system
Information Sector (FAT32 only)
More Reserved
Sectors (optional)
FAT1FAT2
(Duplicated)
Root Folder
(FAT12/FAT16  only)
Other Folders and All Files
Size in sectors  (number of reserved sectors) (number of FATs) * (sectors per FAT) (number of root entries*32) / (bytes per sector) (number of clusters) * (sectors per cluster)
table take from Wiki - File Allocation Table

Term in FAT32 (FAT Volume Components)
Disk Structures:-
(a) Track (red row)
(b) Geometrical sector / sector
(c) Track sector / Block (purple cell)
(d) cluster







Picture take from Wikpedia

Term Description
Boot SectorContains the BIOS parameter block that stores information about the layout of the volume and the file system structures, as well as the boot code that loads Windows Server 2003.
Bootable partition that stores information about the layout of the volume and the file system  structures
Reserved Sectors The number of sectors that precede the start of the first FAT, including the boot sector.
FAT 1 Original FAT.
FAT 2
(Duplicate)
Backup copy of the FAT.
Root Folder Describes the files and folders in the root of the partition.
Other Folder
and all files
Contains the data for the files and folders within the file system.

FAT32 Structure - Boot Sector (Volume ID / FSInfo)

AddressLengthValueDescription
0x003 byteEB 58 90 (HEX)Jump instruction to boot code, the jump address is 1 byte.
2 allowed forms:-
EB xx 90
E9 xx xx

xx indicates that any 8-bit value is allowed in that byte. What this forms is a three-byte Intel x86 unconditional branch (jump) instruction that jumps to the start of the operating system bootstrap code. This code typically occupies the rest of sector 0 of the volume following the BPB and possibly other sectors. Either of these forms is acceptable. JmpBoot[0] = 0xEB is the more frequently used format.
0x038 byteMSDOS5.0 OEM ID
BIOS Parameter Block - 53 byte 
0x0B2 byte512Bytes per sector
only support value 512, 1024, 2048 or 4096.

maximum compatibility format is 512.
0x0D1 byte2^n
n > 0; n < 7
default = 8
Sector per cluster
[Note] Bytes per cluster Never greater than 32k.
Bytes per cluster = Bytes per sector x Sector per cluster
0x0E2 bytex > 1Reserved sectors
This sector value must equal or more than 1. Usually 32 for FAT32
The reserved sector is start from logical address 0x00 is the Boot Sector and usually contains the operating system's boot loader code. A Backup Boot Sector at logical sector 6.
While many other vendors have continued to utilize a single-sector setup (logical sector 0 only) for the bootstrap loader, Microsoft's boot sector code has grown to spawn over logical sectors 0 and 2 since the introduction of FAT32, with logical sector 0 depending on sub-routines in logical sector 2. The Backup Boot Sector area consists of three logical sectors 6, 7, and 8 as well. In some cases, Microsoft also uses sector 12 of the reserved sectors area for an extended boot loader.
0x101 byte2Number of FATs
Always contain the value 2 for any FAT. The FAT 1 will always use by system and FAT 2 as backup.
Windows should able support any FAT that more than 1.
0x112 byte0Root entries (unused)
It use by FAT12 and FAT16.
FAT32, this field must be set to 0.
0x132 byte0Sectors (on small volumes) (unused)
It use by FAT12 and FAT16.
FAT32, this field must be set to 0.
0x151 byte0xF8Media descriptor (hex)
"Fixed" (non-removable) media / harddisk standard value is 0xF8
Removable media frequently used 0xF0.

Byte   Capacity   Media Size and Type
F0     2.88 MB    3.5-inch, 2-sided, 36-sector
F0     1.44 MB    3.5-inch, 2-sided, 18-sector
F9     720 KB     3.5-inch, 2-sided, 9-sector
F9     1.2 MB     5.25-inch, 2-sided, 15-sector
FD     360 KB     5.25-inch, 2-sided, 9-sector
FF     320 KB     5.25-inch, 2-sided, 8-sector
FC     180 KB     5.25-inch, 1-sided, 9-sector
FE     160 KB     5.25-inch, 1-sided, 8-sector
F8     -----      Fixed disk
0x162 byte0Sectors per FAT (small vol.) (unused)
It use by FAT12 and FAT16.
FAT32, this field must be set to 0.
0x182 byte3F 00 (hex)Sectors per track
Sector per track for interrupt 0x13. This field is only relevant for
media that have a geometry (volume is broken down into tracks by
multiple heads and cylinders) and are visible on interrupt 0x13.
This field contains the “sectors per track” geometry value.
0x1A2 byteFF 00 (hex)Heads
Number of heads for interrupt 0x13. This field is relevant as
discussed earlier for BPB_SecPerTrk. This field contains the one based “count of heads”. For example, on a 1.44 MB 3.5-inch floppy drive this value is 2.
0x1C4 byte3F 00
00 00 (hex)
Hidden sectors
The number of sectors on the volume before the boot sector. This value is used during the boot sequence to calculate the absolute offset to the root directory and data areas. This field is generally only relevant for media that are visible on interrupt 0x13. It must always be zero on media that are not partitioned.
Count of hidden sectors preceding the partition that contains this FAT volume. This field is generally only relevant for media visible on interrupt 0x13. This field should always be zero on media that are not partitioned. Exactly what value is appropriate is operating system specific.
0x204 byte1D 91
11 01 (hex)
Sectors (on large volumes)
Contains the total number of sectors in the FAT32 volume.
FAT32 Section (FAT32 only)
0x244 byte
Sectors per FAT
This value can use to determine where the root directory begins and user data area of the volume begins based on the number of entries in the root directory. This cannot be zero 0x00
0x282 byte00 00 (hex)Extended Flags (FAT mirroring disabled?)
Bit 0-3 -- Zero-based number of active FAT. Only valid if mirroring is disabled.
Bit 4-6 -- Reserved
Bit 7 -- 0 -- FAT is mirrored at runtime into all FATs.
            1 -- One FAT is active, it is the one referenced in bit 0-3
Bit 8-15 -- Reserved.
0x2A2 byte00 00 (hex) Version (usually 0)
High byte is major revision number, Low byte is minor revision number. This support the ability to extend the FAT32 media type in the future without worrying about old FAT32 drivers mounting the volume.
0x2C4 byte2Root dir 1st cluster
This value is first cluster of the root directory. Usually its value is 2 but not required to be 2.
0x302 byte1File System Information Sector Number
The sector number of the File system Information (FSInfo) structure in the reserved area of the FAT32 volume. The value is typical 1. A copy of the FSInfo structure is kept in the Backup Boot Sector, but it is not kept up-to-date.
0x322 byte6Backup boot sector
This value cannot be zero (0x00), but typical value is 6. No other value is recommended.
0x3412 byte00 00 00
00 00 00
00 00 00
00 00 00
Reserved
Reserved space for future expansion. The value of this field must always be zero.
Extended BPB Fields for FAT32 Volumes
0x401 byte80Physical Drive Number
0x411 byte0x00Reserved. 
Always set to zero
0x422 byte29Extended Boot Signature
A field that must have the value 0x28 or 0x29 to be recognized by Windows Server 2003.
0x434 bytexx xx xx xxVolume serial number
A random serial number that is created when a volume is formatted and that helps to distinguish between disks.
0x4711 byteNO NAMEVolume Label
A field that was once used to store the volume label. The volume label is now stored as a special file in the root directory.
0x528 byteFAT32System ID
0x1FE2 byte55 AA (hex)Signature

FAT 1 and FAT 2 Structure
FAT32 able handle larger cluster and make each cluster can be made smaller for particular volume, increasing the efficiency of data storage. FAT32 contains 4 bytes per cluster in the file allocation table. A FAT32 volume must have at least 65,527 clusters.

Logical sector
FAT 1 start location = Reserved Sectors
FAT 2 start location = Reserved Sectors + Sectors per FAT
FirstRootDirSectors = Reserved Sectors + (Number of FATs * Sectors per FAT)
Cluster number of FirstRootDirSectors have to refer value of "Root dir 1st cluster 0x2C"
RootDirSCount =
FirstDataSector =  Reserved Sectors + (Number of FATs * Sectors per FAT) + RootDirSCount

Physical sector
FAT 1 start location = (PartitionEntry->StartingSector value) + Reserved Sectors
FAT 2 start location = (PartitionEntry->StartingSector value) + Reserved Sectors + Sectors per FAT
FirstRootDirSectors= (PartitionEntry->StartingSector value) + Reserved Sectors + (Number of FATs * Sectors per FAT)
FirstDataSector = (PartitionEntry->StartingSector value) + Reserved Sectors + (Number of FATs * Sectors per FAT) + FirstRootDirSectors
[Note] Please read Master Boot Record to understand PartitionEntry->StartingSector value
             Normally, it is 0x1C6 = (0x01BE + 0x08)


Root Directory Structure
OffsetSizeDescription
0x007 ByteShort File Name
0x00 address = 00 -- Never used
                         E5 -- Erased
                         05 -- special file name like japan or china word.
                      other -- FileName
0x083 byteFile Extension
0x0B1 byteFile Attribute (refer Attribute Value)
when value = 0F, there is long file name.
0x0C1 byteReserved for use by Windows NT.
0x0D1 byteMilisecond stamp at file creation time.
0x0E2 byteTime file was created
0x102 byteDate file was created
0x122 byteLast access date. There is no last access time.
0x142 byteHigh word of cluster (FAT32 only)
0x162 byteTime of last write
0x182 byteDate of last write
0x1A2 ByteFile Cluster Low
0x1C4 ByteFile Size (Zero for a directory)

FAT Date time conversion
Time conversion

<------- 0x0F --------> <------- 0x0E -------->
15 14 13 12 11 10 09 08 07 06 05 04 03 02 01 00
h  h  h  h  h  m  m  m  m  m  m  s  s  s  s  s

The FAT time resolution is 2 second, mean,
s s s s s hex => decimal x 2 = display second.

Date conversion

<------- 0x10 --------> <------- 0x11 --------> 15 14 13 12 11 10 09 08 07 06 05 04 03 02 01 00 y y y y y y y m m m m d d d d d

Attribute Value
Attrib BitFunctionLFNComment
0 (LSB)Read Only1Should not allow writing
1Hidden1Should not show in dir listing
2System1File is operating system
3Volume ID1Filename is Volume ID
4DirectoryxIs a subdirectory (32-byte records)
5ArchivexHas been changed since last backup
6Ununsed0Should be zero
7 (MSB)Ununsed0Should be zero

Long File Name for FAT32
Example:-

Condition
1. The drive contain a file that mean testing12345678.ASM

The Long File Name format
1. File name initial with 0x42, but this is last of the file name will display at 1st row.
    The file name is convert into Unicode. 78.AS => 7 8 . A S (Alternate with space.)
    follow by 0x0F 0x00, check sum and M .[after M have space because Unicode]
   [Note] The 0x0F is Attribute.
              The initial value 0x42 will increase, if your file name length is long.
              It depend on number of file entry that using.
   Check sum algorithm:-

  1. The ASCII value of the first character is the base sum.
  2. Rotate the sum bitwise one bit to the right.
  3. Add the ASCII value of the next character to the sum.
  4. Repeat step 2 and 3 until all 11 characters has been added.


2. Follow by 0x0000 0xFFFF ..... 0x0000 0xFFFF 0xFFFF
    The end of file name will fill with 0x0000 0xFFFF
    The ... is 0xFFFF until Address 0xXXA the code must fill with 0x0000 0xFFFF 0xFFFF
3. Initial with 0x01 follow by file name t e s t i
    follow by 0x0F 0x00, check sum and "n"
    The 0x01 is the file name allocation. It can be 0x02, 0x03, and so on.
4. other file name - g 1 2 3 4 + check sum + 5 6
5. As Normal directory format (refer Root Directory Structure)

Reading the File Base on FAT pointer



Base on the example on Top

Example 1
1. The File testing12345678.asm located at cluster  0x00 00 00 08 (0x400134 + 0x40013A)
    The program have to go cluster 8 to read data, read all data until end of cluster.
2. When reach end of cluster, program have to jump to FAT1.
    [Note] You will see FAT1 and FAT2 is same.
               The sector 0 and 1 0x0030D400 - 0x0030D407 is reserved (boot sector)
    4 bytes of FAT value form 1 cluster in FAT32 file system.
3. Since you currently point to cluster 8, then you need refer FAT 1 cluster 8 to get next location.
    FAT 1 cluster 8  logical address is 0x0030D420 - 0x0030D423.
    The value is 0xFF FF FF 0F, means end of file.

Example 2
1. Now program are reading cluster 5 until the end. The program will switch the FAT 1 to find the next cluster.
2. The FAT 1 cluster 5 show value 0x00 00 00 06, the program will go to sector 6 continue reading file until the end. Again, switch it will switch the FAT 1 to find the next cluster. This time is FAT 1 cluster 6 because now it active on cluster 6.
3. The FAT 1 cluster 6 show value 0x00 00 00 07. repeat the reading process and back to FAT again.
    Until 0xFF FF FF 0F is end of file.

Default cluster sizes for FAT32
The following table describes the default cluster sizes for FAT32.

Volume sizeWindows NT 3.51Windows NT 4.0Windows 7, Windows Server 2008 R2,
Windows Server 2008, Windows Vista,
Windows Server 2003, Windows XP, Windows 2000 
7 MB–16MBNot supportedNot supportedNot supported
16 MB–32 MB512 bytes512 bytesNot supported
32 MB–64 MB512 bytes512 bytes512 bytes
64 MB–128 MB1 KB1 KB1 KB
128 MB–256 MB2 KB2 KB2 KB
256 MB–8GB4 KB4 KB4 KB
8GB–16GB8 KB8 KB8 KB
16GB–32GB16 KB16 KB16 KB
32GB–2TB32 KBNot supportedNot supported
> 2TBNot supportedNot supportedNot supported


This table is take from Microsoft website.

FAT32 Limitation
DescriptionLimit
Maximum file sizeTested: 4 GB minus 1 byte
(2^32 bytes minus 1 byte)
Maximum volume sizeTested: 32 GB (implementation)
Files per volume4,177,920
Maximum number of files
and subfolders within a single folder
65,534 (The use of long file names can
significantly reduce the number of available
files and subfolders within a folder.)


The following limitations exist using the FAT32 file system with Windows operating systems:
  • Clusters cannot be 64 kilobytes (KB) or larger. If clusters were 64 KB or larger, some programs (such as Setup programs) might calculate disk space incorrectly.
  • A volume must contain at least 65,527 clusters to use the FAT32 file system. You cannot increase the cluster size on a volume using the FAT32 file system so that it ends up with less than 65,527 clusters.
  • The maximum possible number of clusters on a volume using the FAT32 file system is 268,435,445. With a maximum of 32 KB per cluster with space for the file allocation table (FAT), this equates to a maximum disk size of approximately 8 terabytes (TB).
  • The ScanDisk tool included with Microsoft Windows 95 and Microsoft Windows 98 is a 16-bit program. Such programs have a single memory block maximum allocation size of 16 MB less 64 KB. Therefore, The Windows 95 or Windows 98 ScanDisk tool cannot process volumes using the FAT32 file system that have a FAT larger than 16 MB less 64 KB in size. A FAT entry on a volume using the FAT32 file system uses 4 bytes, so ScanDisk cannot process the FAT on a volume using the FAT32 file system that defines more than 4,177,920 clusters (including the two reserved clusters). Including the FATs themselves, this works out, at the maximum of 32 KB per cluster, to a volume size of 127.53 gigabytes (GB).
  • You cannot decrease the cluster size on a volume using the FAT32 file system so that the FAT ends up larger than 16 MB less 64 KB in size.
  • You cannot format a volume larger than 32 GB in size using the FAT32 file system in Windows 2000. The Windows 2000 FastFAT driver can mount and support volumes larger than 32 GB that use the FAT32 file system (subject to the other limits), but you cannot create one using the Format tool. This behavior is by design. If you need to create a volume larger than 32 GB, use the NTFS file system instead.
NOTE: When attempting to format a FAT32 partition larger than 32 GB, the format fails near the end of the process with the following error:
Logical Disk Manager: Volume size too big.

Friday, May 17, 2013

Master Boot Record

Master Boot Record
The Master Boot Record is the same for most of pretty much all Operating System. As long as the platform is following x86 PC architecture (IBM PC). It is located on the first sector of Hard Drive at Cylinder 0, Head 0, Sector 1 (That mean, it start at  physical address 0x00). It is the first piece of code that your computer runs after it has checked all of your hardware (POST) and turned control of loading software over the hard driver. This sector contain hard disk boot loader (ROM BIOS bootstrap routine), where the code can understand by PC BIOS. It aslo contains the partition table, which defines the difference sections of your hard drive. Basically if anything happens to this little 512 byte section, your hard driver is brain dead. This section will change partition table when your doing OS level formatting. Except your doing Fdisk (knows as Low level format) will change the Master Boot Record.

Partition Table
Partition table is a small part of master boot record. It contain the partition start and end physical address.


Master Boot Record and Partition Boot Record in physical address

AddressLengthDescription
0x0000 to 0x01BD 446 bytes Boot Loader Code
0x01BE to 0x01CD 16 bytes Partition entry 1
0x01CE to 0x01DD 16 bytes Partition entry 2
0x01DE to 0x01ED 16 bytes Partition entry 3
0x01EE to 0x01FD 16 bytes Partition entry 4
0x01FE to 0x01FF 16 bytes Boot Record signature (55 AA)


Details of Partition Entry Table

LengthDescription
1Boot indicator (0x00 = Inactive, 0x80 = Active)
3Starting head, cylinder and sector (refer Head, Cylinder and sector)
1File system descriptor (refer Partition Type Listing)
3Ending head, cylinder and sector
4Starting sector (offset to disk start)
4Number of sector in partition


Head, Cylinder and sector
Cylinder Bit 7 to 0 (MSB)Cylinder Bits 9+8Sector Bits 5 to 0Head Bit (LSB)


Partition Type Listing
Table at below only show the Microsoft partition type listing

ValueDescription
0x00Unknown or Nothing
0x0112-bit FAT
0x0416-bit FAT (Partition Smaller than 32MB)
0x05Extended MS-DOS Partition
0x0616-bit FAT (Partition Larger than 32MB)
0x0B32-bit FAT (Partition Up to 2048GB)
0x0CSame as 0x0B, but uses LBA 0x13 Extensions
0x0ESame as 0x06, but uses LBA 0x13 Extensions
0z0FSame as 0x05, but uses LBA 0x13 Extensions

[Note] LBA - Logical Block Addressing - Uses the Interrupt 0x13 extension built into newer BIOS to access data above the 8GB barrier, or to access strictly in LBA mode, instead of CHS (Cylinder, Head Sector)

Reference
FAT32 Disk structure
FDISK / MBR rewrites the Master Boot Record
Master Boot Record (x86)