Featuring large capacity and low power consumption, Shingled Magnetic Recording (SMR) disks have been widely used to store cold objects by many public clouds, such as AWS Glacier, Google, and Facebook. However, SMR disks are seldom used in enterprise storage, because the co-existence of hot and cold data requires more data access modes and higher performance than the cold storage on public clouds. In the beginning of 2018, Huawei took the lead to use 14 TB SMR disks on its star product, OceanStor 9000 distributed storage. The new combination is especially suitable for video surveillance and backup and archiving scenarios, reducing the space occupation and power consumption by over 40%.
This article will unveil the design and technologies behind the successful application of large-capacity SMR disks on OceanStor 9000.
Background about the SMR Technology
In the Big Data era, the volume of data grows exponentially. IDC predicts that the global data volume will reach 163 ZB in 2025 and the storage capacity will exceed 19 ZB, among which 58% will be stored on HDDs. The storage density of HDDs (the number of bits per unit area) is limited by the law of physics. The Perpendicular Magnetic Recording (PMR) technology used by mainstream HDDs is about to reach its storage density limit. The industry is eager for new technologies to overcome the restrictions and continue to increase capacity steadily.
IDC predicts that the global data volume will reach 163 ZB in 2025
After careful study, Huawei chooses the leading next-generation disk technology, SMR, for its new storage offerings. It overlaps tracks like shingles on the roof to increase the storage density, while keeping existing magnetic heads and media technologies unchanged.
A conventional disk sheet is divided into concentric circular tracks, with certain space in-between. Two magnetic heads (one for write and one for read) are above the medium. Because the magnetic field strength required for writing is higher than that for reading, the width of the write head is greater than that of the read head. The width of the write head determines the number of tracks that each inch of disk can hold.
Comparison between the recording modes of traditional disks and SMR disks
An SMR disk takes advantage of the width difference between read and write heads and overlaps adjacent tracks like shingles. Data is written into the first of overlapping tracks. At the same time, sufficient space is reserved for the narrower read head to read data from the first track. Compared with traditional disks, an SMR disk has more tracks, which increases the storage density and reduces the cost of unit storage capacity. However, because of this feature, an SMR disk only supports sequential write of large blocks, and do not support random write and modification. Therefore, a brand new data management solution is required from the upper-layer file system.
Classification of SMR Disks
The write sequence of an SMR disk can be managed by the hard disk itself or by software on the host side. The former type is called Drive Managed SMR (DM SMR) disks. The later mode exposes the internal structure of an SMR disk to upper-layer software through new interfaces. Based on whether random write is allowed, the later type can be classified into two sub-types: If an SMR disk only allows the upper layer to write data in sequence and does not allow random write, it is called a Host Managed SMR (HM SMR) disk. If an SMR disk does not forbid random write operations, it is called a Host Aware SMR (HA SMR) disk.
| Type | Interface | Random Write | Technical Complexity | Whether the Performance Is Controllable |
| DM | Traditional hard disk interface | Yes | None | Uncontrollable, large fluctuation |
| HA | Dedicated interface of an SMR disk, compatible with traditional hard disk interfaces | Yes | Medium | Partially controllable, medium fluctuation |
| HM | Dedicated interface of an SMR disk | No | High | Controllable and small fluctuation |
The above table summarizes the characteristics of the three types of SMR disks. A DM SMR is easy to use and no change needs to be made to the existing software. However, due to random write, background operations such as data migration and garbage collection are inevitable, leading to violent performance fluctuation. An HM SMR disk requires optimizations to the storage system software (such as distributed file systems), to maximize the performance of SMR disks. HA SMR is a trade-off solution: it provides the highest flexibility and may cause performance fluctuation.
Different from Internet applications, enterprise applications, especially video surveillance services, have strict requirements on the stability and predictability of storage systems. Therefore, Huawei OceanStor 9000 selects HM SMR disks, which are the most complex in terms of technologies, to provide stable performance for enterprise applications.
Challenges of Using SMR Disks in Enterprise Storage
Storage of cold objects on public clouds does not involve modification generally, and the data retrieval time is long. For example, with AWS Glacier, standard retrieval usually takes 3 to 5 hours. In enterprises, most data read and written is files, so there may be many modifications. The response time to read and write requests is usually in milliseconds. For example, in video surveillance scenarios, if the average latency of write requests is longer than 40 ms, the written video data would be lost.
Currently, the capacity of an SMR disk is up to 14 TB, which will grow to 18 TB in the second half of 2018. In comparison, the capacity of a mainstream SATA/NL_SAS disk in the enterprise market is only 8 TB. It would take much longer time for data to be reconstructed on a large-capacity hard disk. For traditional enterprise storage, reconstructing 1 TB data takes about 10 hours. Reconstructing 14 TB data on an SMR disk would take 140 hours (nearly one week). During this period, if another hard disk gets faulty, data would be lost. This is absolutely unacceptable to enterprise applications.
Facing these challenges, storage vendors wanting to use SMR disks in enterprise storage must adapt storage systems to SMR disks, keep user habits and performance unchanged, and ensure high reliability. They cannot simply substitute traditional hard disks with SMR disks. Instead, the entire storage stack needs to be optimized: everything from upper-layer application software to underlying drivers needs to adapt to the new interfaces of SMR disks, to support the sequential write of I/Os, prevent competition conditions and I/O disorder, and achieve higher performance and reliability.
(Contributed by Cui Yuxiang, IT Product Line)
The post Saving 40% Space and Power with Huawei’s New SMR Storage (1) appeared first on Huawei Enterprise Blog.
Source: Huawei Enterprise Blog
—