Deploying the Next Generation of High-Capacity Hard Drives

A Brief History of Storage Architectures

It has been 30 years since a small software company made an agreement with IBM to produce the first operating system for what would become the IBM PC. That company was, of course, Microsoft and the operating system was DOS or PC-DOS, as it became branded by IBM. DOS became the basis for the operating system that would be broadly adopted as the standard in the PC industry and would also serve as the foundation for many aspects of the Microsoft Windows operating system.

In those early days, no one really envisioned the incredible advances in the computer industry that would lead ultimately to today’s super-fast processers, massive solid state memory and huge hard drives, now commonly available in sizes up to 2 TB. Some of the basic design decisions made in the original computer architecture, both hardware and software, left the industry with inherent limitations. One of those limitations is impacting the storage industry today: the ability to address hard drives that exceed 2.1 TB capacities.

One of the most fundamental elements of storage architecture is the sector. A sector is the smallest physical block of data represented on a hard drive. Over 30 years ago, the sector was defined as 512 bytes in length and that definition has persisted to this day. Today there is a move to transition to a larger 4K sector (also called Advanced Sector Format), but this transition will not be implemented at the host level (e.g. hard drive controllers, OS, BIOS) for a number of years. In the meantime, the storage industry must still contend with a base sector size of 512 bytes, despite the growing need for more capacity.

It turns out that another key architectural decision was made relating to hard drive sectors. This was connected with the space set aside to address sectors. Each sector is assigned a unique address that defines the data's location on a hard drive. This is called a logical block address (LBA). Back in the late 1970s and early 1980s, no one could fathom a hard drive even approaching 1 TB, so limiting the LBA range to 2.1 TB was thought to be more than enough. As a result, operating systems, BIOS controllers, HDD controllers and device drivers use the same basic limitation of 2.1 TB for the maximum size of a hard drive or logical storage device.

Now that the industry is on the verge of releasing hard drives that exceed this 2.1 TB capacity limitation, attention is being focused on how to deploy this new generation of high-capacity storage solutions in light of these historical architectural design limits. It is best to do this in the context of looking at two distinct market segments: the desktop PC and workstation segment, and the enterprise segment focusing on multi-drive storage arrays and servers.

The Desktop Market

In evaluating how to deploy hard drives larger than 2.1 TB in a desktop environment, three distinct installation conditions must be considered:

1. Deployment of any hard drive with a native capacity larger than 2.1 TB using Microsoft Windows XP or older, or using another operating system that is not capable of long LBA addressing
2. Deployment of a non-bootable hard drive with a native capacity larger than 2.1 TB using an operating system that is capable of long LBA addressing
3. Deployment of a bootable drive where the native drive size is larger than 2.1 TB.

Condition One

Unfortunately, there is no working solution for using hard drives larger than 2.1 TB with any operating system that does not support long logical block addresses, often referred to as long LBA addressing. This includes Windows XP, which still has a sizable installed base. Long LBA addressing extends the number of bytes used in a Command Descriptor Block (CDB) to allow access to an LBA range that exceeds the 2.1 TB limitation. A CDB is simply a data structure that is used to format data passed between host computers and hard drives. A key element of this data is the LBA, which tells the hard drive which specific sector of data is being addressed.

Operating systems without this long LBA addressing capability cannot recognise a hard drive larger than 2.1 TB. In fact, using a hard drive larger than 2.1 TB in a Windows XP system can produce unpredictable results, depending on the BIOS used. It may recognise only the capacity up to the 2.1 TB limit or it may in fact only recognise the capacity over the 2.1 TB limit. So, a 2.5 TB hard drive might be recognised by Windows XP as a 400 GB drive or a 2.1 TB drive, but not correctly as a 2.5 TB drive. Using hard drives with a native capacity larger than 2.1 TB requires Windows Vista, Windows 7 or another long LBA capable OS.

Condition Two

We have now established that the basic operating system needs to support long LBA addressing in order to use a hard drive larger than 2.1 TB. If the desired use of this drive is as a non-bootable drive, only one more requirement remains: the drive must be partitioned using a GUID Partition Table (GPT).

A GPT was defined as part of a more comprehensive specification originally launched by Intel with the purpose of creating a replacement for the original PC BIOS design. This standard was named the Extensible Firmware Interface (EFI) and the specification is now managed by the Unified EFI Forum (UEFI). In addition to other enhancements, a GPT allows for a much larger LBA addressing scheme, which enables the use of hard drives in excess of 2.1 TB in size. Furthermore, the legacy standard MBR (Master Boot Record) will result in the same limitations discussed earlier and limits the hard drive capacity to 2.1 TB.

Condition Three

Clearly, the use of a single hard drive in a desktop PC configuration is very common. In this case, the hard drive must be bootable. Using a hard drive larger than 2.1 TB for a bootable configuration has the same requirements as described above in condition two, with two additional requirements.

First, in order to overcome problems in deploying hard drives larger than 2.1 TB (for bootable drives), it is important to look at the legacy PC BIOS. The PC BIOS standard that has been used for many years also has the same fundamental limitation in not being able to address a hard drive larger than 2.1 TB. Today the only solution to this limitation is the use of a new BIOS standard, also part of the work of the UEFI Forum. The UEFI BIOS has a number of extensions that go beyond the capabilities of the older PC BIOS system. Among these is the capability to address hard drives larger than 2.1 TB. To date, however, the use of a UEFI BIOS in the desktop market is rare. Nevertheless, with the introduction of a new generation of high-capacity hard drives, adoption of UEFI BIOS systems will accelerate.

The second requirement for deployment of a hard drive larger than 2.1 TB is the use of a compatible hard drive controller driver. One of the most popular drivers used in the PC marketplace is the Intel Matrix Storage driver. This driver, which is commonly a component of the Windows Vista and Windows 7 OS configuration, also shares the limitation of not properly addressing hard drives larger than 2.1 TB. Intel is planning an update to this driver in the near future. In the meantime, the standard Windows HDD driver included in the Windows Vista and Windows 7 installations solves this issue.

Servers and Storage Arrays

Fundamentally, the requirements described for the standard desktop market in the use of hard drives larger than 2.1 TB also apply to the server and storage array markets. The most significant difference in these two markets is the dominant use of Host Bus Adapters (HBAs) or RAID controllers to manage and control multiple hard drives, common in enterprise solutions.

The use of an HBA or RAID controller simplifies the adoption of these new larger hard drives as the addressing and control issues are essentially all managed by the HBA or RAID controller. The details of the sizes and types of drives behind the controller are, unfortunately, hidden from the OS.

Furthermore, these advanced storage solutions typically use a very small hard drive as the primary boot drive, while depending on the HBA or RAID controller to manage the remaining hard drives installed as part of the total solution. Seagate recommends checking with your RAID or HBA controller manufacturer to explore the use of hard drives larger than 2.1 TB, ensuring that they are compatible. Several controller manufacturers are limiting this support to the latest 6 Gb/s SAS and SATA controllers, which, in turn, may require an upgrade if you are currently using a 3 Gb/s controller.

If you choose to run a hard drive larger than 2.1 TB off an existing legacy controller without the long LBA addressing capability, you must mode select the drive to de-stroke the drive capacity to match the capabilities of your legacy controller.

Summary

There are near-term obstacles to overcome to allow broad-based adoption of hard drives larger than 2.1 TB in the desktop environment, particularly in the case of bootable drives. The adoption of these new high-capacity drives in the enterprise market has fewer and simpler limitations. While these conditions may seem complex in the context of a detailed explanation, a concise requirements summary is shown in Table 1.

Second Drive (non-bootable) Solution Desktop, Workstation

Primary Drive (bootable) Solution Desktop, Workstation

Storage Server or Storage Array Solutions

Call to Action

Even small businesses need terabytes of storage today. As storage demands continue to grow and the storage market continues to make improvements in capacity, performance and cost efficiencies, larger hard drives will continue to be a vital part of these improvements. This is particularly true in the enterprise markets where data centres and cloud computing readily adopt high capacities where they can be deployed quickly with significant leverage in driving operational efficiencies and reducing storage costs. The next generation of large hard drives, extending past the 2.1 TB limit for legacy systems, will begin delivery in the summer or autumn of 2010. In order to minimise integration complexities and to allow customers to take advantage of the benefits of these new larger hard drives, system builders, integrators and storage solution providers need to act now.

If your product roadmaps focus on storage arrays and servers that depend on RAID controllers and HDD host bus adapters, contact your vendor and ask about their plans for high-capacity HDD support above 2.1 TB.

If the solutions you develop are more focused on desktop, workstation or server solutions that depend on motherboard-equipped HDD controllers and OS support, you should begin these conversations with your component system suppliers. Intel and Microsoft are key enablers and they are already engaged in the discussions to solve these limitations. However, your voice is important in establishing a sense of urgency for these suppliers, as well as the broader ecosystem of suppliers who can help implement the solutions to overcome these legacy limitations for HDD support above 2.1 TB.

Working together, we can continue to bring high-capacity, high-performance computing and storage solutions to market and continue to drive productivity and bring economic benefits to our industry.