Transition to Advanced Format 4K Sector Hard Drives

Overview

A change has come in the hard drive industry. As storage densities increased dramatically over the years, one of the most elemental aspects of hard drive design, the logical block format size know as a sector, had remained constant.

Around 2010, hard drive companies began migrating away from the legacy sector size of 512 bytes to a larger, more efficient sector size of 4096 bytes, generally referred to as 4K sectors and now referred to as the Advanced Format by IDEMA (The International Disk Drive Equipment and Materials Association).

This paper provides the context for this migration, as well as pointing out the long-term benefits to customers and potential pitfalls to be avoided in the move from 512 bytes to 4K sectors.

Background

For over 30 years, data stored on hard drives has been formatted into small logical blocks called sectors, with a legacy sector size of 512 bytes. Many aspects of modern computer systems still have design assumptions based on this format standard.

The legacy sector format contains a Gap section, a Sync section, an Address Mark section, a Data section, and Error Correction Code (ECC) section (Figure 1).

Figure 1. Legacy Sector Layout on Hard Drive Media

The structure of this sector layout was designed as follows:

• Gap section: The gap separates sectors.
• Sync section: The sync mark indicates the beginning of the sector and provides timing alignment.
• Address Mark section: The address mark contains data to identify the sector’s number and location. It also provides status about the sector itself.
• Data section: The data section contains all of the user’s data.
• ECC section: The ECC section contains error correction codes that are used to repair and recover data that might be damaged during the reading or writing process.

This low-level format served the industry well for many years. However, as hard drive capacities increased, sector size increasingly became a limiting design element in improving hard drive capacities and error correction efficiency. For example, comparing the sector size to the total capacity of earlier to more recent hard drives, you can see that the sector resolution has become extremely small. The resolution of the sector (the ratio of sectors as a percent of total storage) has become very fine and increasingly inefficient (Table 1).

Very fine resolution is good when managing small, discrete amounts of data. However, applications common in modern computing systems manage data in large blocks, much larger in fact than the legacy 512-byte sector size.

More importantly, the small 512-byte sector has consumed a smaller and smaller amount of space on the hard drive surface as areal densities have increased. This is a problem in the context of error correction and the risk of media defects. In Figure 2, for example, the data in a hard drive sector is consuming smaller areas, making error correction more challenging because media defects of the same size can damage a higher percentage of the total data payload and, therefore, require more error correction strength.

Figure 2. Media Defects and Areal Density

A 512-byte sector can typically correct a defect of up to 50 bytes in length. Today’s hard drives have begun to push the limits on error correction with leading areal densities. Consequently, the migration to larger sectors within the hard drive industry is a fundamental need relative to gaining improvements in error correction and achieving format efficiencies.

The Transition to 4K Sectors (Advanced Format)

The storage industry has been planning the transition to larger sector hard drive formats for years; significant work at Seagate and with our peers in the hard drive industry dates back to 2005 (Figure 3). In December of 2009, through a coordinated effort within IDEMA, Advanced Format was nominated and approved as the name for the standard of 4K-byte sectors. In addition, all hard drive manufacturers committed to shipping new hard drive platforms for desktop and notebook products with the Advanced Format sector formatting beginning in January of 2011. Even before that date, Advanced Format drives began to enter the market. Seagate first shipped large sector drives to OEM customers and in branded retail products.

Figure 3. Key Milestones in the Development of the Advanced Format Standard

The Long-Term Benefit to 4K Sectors

As all hard drive manufacturers agreed to transition to the Advanced Format sector design by January 2011, the industry has adapted to and embraced this change to minimize potential negative side effects. While short-term benefits to end users are not dramatic in terms of immediate capacity increases, the migration to 4K-sized sectors have most definitely provided quicker paths to higher areal densities and hard drive capacities, as well as more robust error correction.

Improved format efficiency by reducing the amount of space used for error correction code

In Figure 4, the legacy 512-byte sector layout is shown, where each 512-byte sector has non-data-related overhead of 50 bytes for ECC and another 15 bytes for the Gap, Sync, and Address Mark sections. This yields a sectorized¹ format efficiency of about 88% (512/(512 65)).

Figure 4. Legacy 512-Byte Sector Layout

The new Advanced Format standard makes the move to a 4K-byte sector, which essentially combines eight legacy 512-byte sectors into a single 4K-byte sector (Figure 5).

Figure 5. Advanced Format: 4K-Byte Sector Layout

The Advanced Format standard uses the same number of bytes for Gap, Sync, and Address Mark, but increases the ECC field to 100 bytes. This yields a sectorized¹ format efficiency of 97% (4096/(4096 115)), almost a 10% improvement.

Over time, these format efficiencies pay off, helping to yield higher capacity points while also improving data integrity.

Reliability and Error Correction

While the physical size of the sectors on hard drives has shrunk, taking up smaller and smaller amounts of space, media defects have not. Consider Figure 6, which displays images of what we would consider very small objects. In relation to the fly height of a hard drive read/write head, these objects are relatively large. Microscopic particles much smaller than those in this diagram can create media defects on a hard drive.

Figure 6. Small-Scale Representation of Hard Drive Fly Height

The larger 4K sector in the Advanced Format standard approximately doubles² the size of the ECC block from 50 bytes to 100 bytes, providing a much needed improvement in error correction efficiency and robustness against particles and media defects.

Together, the benefits of improved format efficiency and more robust error correction make the transition to 4K sectors well worth the effort. Properly managing this transition to capture the long-term benefits with minimal side effects has been a key focus for the hard drive industry.

Understanding the Impacts to the 4K Transition

As noted earlier, there are many aspects of modern computing systems that continue to assume that sectors are always 512 bytes. To transition the entire industry over to the new 4K standard and expect all of these legacy assumptions to suddenly change is simply not realistic. Over time, the implementation of native 4K sectors, where both host and hard drive exchange data in 4K blocks, will take place. Until then, hard drive manufacturers will implement the 4K sector transition in conjunction with a technique called 512-byte sector emulation.

512-Byte Sector Emulation

The introduction of 4K-sized sectors had depended heavily on 512-byte sector emulation. This term refers to the process of translating from the 4K physical sectors used in Advanced Format to the legacy 512-byte sectors expected by host computing systems.

The 512-byte emulation is acceptable in that it does not force complex changes in legacy computing systems. However, it carries the potential for negative performance consequences, particularly when writing data that does not neatly correspond to eight translated legacy sectors. This becomes clear when considering the reading and writing process required by 512-byte emulation.

Emulated Read and Write Processes

To read data from a 4K sector formatted drive in 512 emulation mode, the process is very straightforward, as shown in Figure 7.

Figure 7. Potential Read Sequence for 512-Byte Emulation

The process of reading the 4K block of data and reformatting the specific 512- byte virtual sector requested by the host computer is performed in the drive’s DRAM memory and does not measurably impact performance.

A write process can be more complicated, particularly when data the host computer attempts to write is a subset of a physical 4K sector. In these cases, the hard drive must first read the entire 4K sector containing the targeted location of the host write request, merge the existing data with the new data and then rewrite the entire 4K sector (Figure 8).

Figure 8. Potential Write Sequence for 512-Byte Emulation

In this instance, the hard drive must perform extra mechanical steps in the form of reading a 4K sector, modifying the contents, and then writing the data. This process is called a read-modify-write cycle, which is undesirable because it has a negative impact on hard drive performance. Minimizing the probability and frequency of read-modify-write instances is the most important aspect of making the transition to 4K sectors smooth and painless.

Read-Modify-Write Prevention

1. Write requests that are misaligned because of logical-to-physical partition misalignment
2. Write requests smaller than 4K in size

Aligned Versus Unaligned Hard Drive Partitions

Up to now we have not discussed how host systems and hard drives communicate the location of sectors on the media. It’s time to introduce the Logical Block Address (LBA).

Each 512-byte sector is assigned a unique LBA, from zero (0) to the number required based on the size of the disk. The host requests a specific block of data using the assigned LBA. When the host requests to write data, an LBA address is returned at the end of the write telling the host where the data is located. This becomes important in the transition to 4K sectors since there are eight different possibilities for where the host LBA starts.

When LBA 0 is aligned to the first virtual 512-byte block in the 4K physical sector, the logical-to-physical alignment condition for 512-byte emulation is termed Alignment 0. Another possible alignment is when LBA 0 is aligned to the second virtual 512-byte block in the 4K physical sector. This situation is termed Alignment 1 and is shown in comparison to the Alignment 0 condition in Figure 9. There are six additional possibilities for unaligned partitions that can result in read-modify-write events similar to the Alignment 1 condition.

Figure 9. Alignment Conditions

Alignment 0 conditions work very well with the 4K sectors in the Advanced Format standard. This is because a hard drive can easily map eight contiguous 512-byte sectors into a single 4K sector. This is accomplished by storing 512-byte write requests in the hard drive’s cache until enough contiguous 512-byte blocks are received that form a 4K sector. Since modern computing applications deal with chunks of data that are typically larger than 4K, runts are extremely rare. However, the Alignment 1 situation is another matter.

When hard drive partitions are created that result in an unaligned condition, as shown in Figure 9, this results in read-modify-write cycles that can slow hard drive performance. This is the key condition that must be avoided in the implementation of Advanced Format hard drives, and is discussed later.

Small Writes

In modern computing applications, data such as documents, pictures, and video streams are much larger than 512 bytes. Therefore, hard drives can store these write requests in cache until there are enough sequential 512-byte blocks to build a 4K sector. As long as hard drive partitions are aligned, the hard drive can easily map 512-byte sectors into 4K sectors without any performance penalties. There are, however, certain low-level processes that can force a hard drive to deal with runt situations that are not associated with unaligned partitions. These occur in rare instances where the host makes discrete write requests that are actually smaller than 4K. These are typically OS-level activities dealing with the file systems, journaling or similar low-level activities. Generally, these occur in small enough numbers so that overall performance is not significantly impacted. Still, it’s recommended that system designers consider proper modifications to any of these processes to maximize performance when making the 4K transition.

Preparing for and Managing the 4K Transition

Now that we understand the benefits of migrating to 4K sectors, as well as the potential impacts to performance, it is time to examine how best to manage this transition. This topic is best discussed in the context of the two most popular operating systems deployed in modern computing: Windows and Linux.

Managing 4K Sectors in the Windows Environment

The single most important aspect of managing a transition to 4K sectors is related to the alignment issues described above. Advanced Format drives work well in an Alignment 0 condition, where the physical-to-logical starting position is equal. Alignment conditions are created when the hard drive partition(s) is created. Partitions are created by software that falls into two general categories:

1. Windows OS releases
2. Hard drive partitioning utilities

In the case of partitions created with the Windows OS, there are three Windows releases that warrant discussion: Windows XP, Windows Vista, and Windows 7. Microsoft was involved with the community planning the transition to larger sectors. Consequently, they released 4K-sector-compatible software beginning with Windows Vista Service Pack 1. Software that creates partitions of Alignment 0 (those that work well with the Advanced Format standard) is referred to as “4K aware.” Table 2 describes the situation relative to current generations of the Microsoft Windows OS.

Clearly, computer systems shipping with the latest versions of Windows are not a concern relative to the use of Advanced Format hard drives. However, for systems that are still using Windows XP or Windows Vista, pre- Service Pack 1, there is significant risk of reduced performance relative to the partitions created by the OS.

In addition to the potential for unaligned partitions created with older versions of the Windows OS, there are also a number of software utilities that are widely used by system builders, OEMS, value-added resellers, and IT managers that can result in unaligned partitions. In fact, it is more common for partitions to be created with these types of utilities rather than with the Windows OS itself. Thus the risk of creating unaligned partitions, and therefore an environment that can result in degraded hard drive performance for drives using 4K sectors, is significant. To complicate the matter, hard drives shipped in systems today typically consist of multiple hard drive partitions. This means that each partition on the hard drive must be created with 4K-aware partitioning software to make sure proper alignment and performance is ensured. Figure 10 shows the potential results of creating multiple hard drive partitions with software that is not 4K aware.

Figure 10. Multiple Partitions and Alignment Conditions

Dealing With Unaligned Conditions

There are three potential methods to avoid and/or manage an unaligned condition that has the potential to impact hard drive performance.

1. Use a new version of the Windows OS or work with your partitioning utility provider to get a 4K-aware version of the software.
2. Use a hard drive utility to realign disk partitions.
3. Depend on your hard drive supplier to manage performance regardless of the alignment condition.

Using a 4K-aware version of Windows to create hard drive partitions is a simple and straightforward method to avoid unaligned conditions. Vendors of software utilities that create hard drive partitions should be able to tell you if 4K-aware versions are available. If so, migrate to these versions to avoid further concern.

Some hard drive manufacturers are dealing with this issue by offering utilities that examine existing hard drive partitions and realign as needed. This alternative takes additional time and adds steps to the system building or upgrading process.

Ultimately, hard drive manufacturers will develop more sophisticated methods to manage unaligned conditions while avoiding negative performance impacts.

As the transition to Advanced Format drives continues, all of these methods play a role in maximizing the industry benefits while also avoiding any potential performance impacts.

Managing 4K Sectors in the Linux Environment

The key strategies in managing the transition to 4K sectors in a Windows environment also apply to Linux. Most Linux system users have access to the source code, giving them the ability to customize the OS to fit their specific needs. This provides an opportunity to proactively update their Linux system to properly manage Advanced Format hard drives.

Properly creating disk partitions that are well-aligned to Advanced Format drives and minimizing small system-level writes that can generate runts independently of alignment issues can largely be avoided by making modifications to your Linux system.

Changes have been made to both the Linux kernel and utilities to support Advanced Format drives. These changes ensure that all partitions on Advanced Format drives are properly aligned on 4K sector boundaries. Kernel support for Advanced Format drives is available in kernel versions 2.6.31 and above. Support for portioning and formatting Advanced Format drives is available in the following Linux utilities:

Fdisk: GNU Fdisk is a command line utility that partitions hard drives. Versions 1.2.3 and above support Advanced Format drives.

Parted: GNU Parted is a graphical utility for partitioning hard drives. Versions 2.1 and above support Advanced Format drives.

Conclusion

The industry transition away from the legacy 512-byte sector is a reality. Hard drive manufacturers agreed to adopt the Advanced Format standard no later than January 2011 for new models shipping into the laptop and desktop market segments.

Hard drive engineers continue to drive improved areal densities and more robust error correction. Consumers benefit as hard drives continue to offer higher capacities, lower costs per gigabyte, and continued reliably levels expected by hard drive technology.

The key to a smooth transition has been a well-educated storage community, so that potential performance pitfalls can be avoided. The most critical aspect of a smooth and successful transition to 4K sectors used in Advanced Format has been to promote the use of 4K-aware hard drive partitioning tools. As a system builder, OEM, integrator, IT professional, or even an end user who is building or configuring a computer, be sure to:

• Use Windows Vista (Service Pack 1 or later), Windows 7 or later versions to create hard drive partitions.
• When using third-party software or utilities to create hard drive partitions, check with your vendor to make sure they are updated and confirmed to be 4K aware.
• If you have customers who commonly re-image systems, encourage them to make sure their imaging utilities are 4K aware.
• If you are using Linux, check with your Linux vendor or your engineering organization to make sure your system has adopted the changes to become 4K aware.
• Check with your hard drive vendor for any other advice or guidance on using Advanced Format drives in your systems.

Together with our industry colleagues and customers, we continue to make the transition to Advanced Format 4K sectors smooth and efficient, leveraging the long-term potential benefits for the entire storage industry.

Footnotes

1 Sectorized format refers strictly to the data sectors and does not take into account overhead associated with servo data and other sector layout inefficiencies.

2 Not every implementation of a 4K sector exactly doubles the ECC bytes when moving from 512-byte to 4K sectors.