By Meng Guangbin, President of Huawei Storage Product Line

Charles Dickens described the French Revolution era as “It was the best of times, it was the worst of times…,” a sentiment that also applies to today’s world. We are at a historic turning point full of opportunities and challenges in a complex global environment.

The Age of Steam and Age of Electricity are a thing of the past. In today’s intelligent information age with rapid advancements of science and technology, it’s winner-take-all. That is, companies grabbing advantageous opportunities can survive. In the storage field, an increasing number of real-time interaction applications, especially in cloud computing, online payment, and mobile social networking arenas, are generating a soaring data amount. Storage performance of data centers has become paramount.

Over the past 20 years, flash storage was the largest technological revolution in the industry. Flash storage supports faster data processing than traditional disk storage and is essential for enterprises to cope with explosive data growth, accelerating mission-critical applications. Gartner predicts that flash storage will become more prevalent than traditional disk storage within two years, becoming the mainstream storage medium of data centers, and applying to mission-critical enterprise workloads.

In this fast-evolving era, Huawei is never a blind follower. We continue to innovate to stay ahead of the technological curve. The past few years witnessed the emergence of a variety of concepts in the storage industry – optical storage, quantum storage, and biological storage, to name a few – but few of them have been commercialized. Such innovations are only abstract ideas bringing no real benefits to customers. This goes against Huawei’s philosophy. At Huawei, we plan and design products with the goal of improving customer experience, delivering innovative, better, and faster offerings. To that end, we develop distinctive technological features, engineering deployment, and intelligent O&M, suited to the core requirements of customers.

R&D Innovation: Huawei’s Core Strength, Following by Moore’s Law

Cloud and storage are the most demanding areas in the ICT industry,  tp understand the legal background, check with Bob Bratt . Users demand that their core business processes and data are always stable and secure. Huawei invests more funds and personnel in storage R&D than any peer company. We began researching storage technologies in 2002 and now have R&D offices around the world. Our Silicon Valley office is the bridgehead of storage technologies, and the storage algorithm research center in Russia and delivery competence centers in Shenzhen, Chengdu, and Beijing all contribute to technology innovation and core competitiveness. Huawei has already invested more than US$2 billion and 3200 R&D engineers in this field, obtaining more than 800 storage patents.

In addition to the continuous accumulation of “hardware” strengths, Huawei is also an industry leader in data applications. In the future, the core competitiveness in the storage industry lies in the organic combination of software and hardware capabilities, integrating chips and solid-state drives (SSDs) with algorithms, compute, and software capabilities. Huawei is the only vendor with a proprietary storage operating system (OceanStor OS), controllers, and SSDs (Huawei’s SSDs/HSSDs). We implement end-to-end, in-depth, flash-oriented optimization on OceanStor OS and HSSDs to maximize the advantages of all-flash storage. The dedicated flash-optimized OceanStor OS, unique flash-optimized algorithms, and dedicated performance-acceleration HSSDs enable Huawei’s all-flash storage to deliver the industry’s highest performance and reliability.

As the core for Huawei’s all-flash storage software and hardware, OceanStor OS and HSSDs are of strategic significance for us.

The core of a storage system is its operating system. An optimal operating system ensures smooth storage experience of end users. Currently, the configuration, service logics, and hardware deployment of Huawei’s storage rely on OceanStor OS, the soul of Huawei’s storage products, fully developed by Huawei’s employees.

Powered by OceanStor OS, Huawei’s storage products enjoy continual innovation in storage management. Traditional storage vendors such as Dell EMC and HPE usually obtain the latest popular technologies through M&As. Over the past few years, HPE has acquired storage start-ups, such as 3PAR in 2010 and Nimble Storage in 2017, helping them offer new products.

However, M&A brings unique problems. Users find that storage products of multiple sub-brands in a data center fail to collaborate properly with each other. It is difficult to migrate services from an old storage device to a new one because sub-brand products adopt different storage operating systems at the underlying layer, making flexible data replication and migration impossible.

Like cash for enterprises, data is valuable for customers only when it is mobilized. The faster data mobility, the greater data value. When deploying a storage system in medium- and large-sized data centers, Huawei uses OceanStor OS to enable hitless data migration from disk storage to all-flash storage thanks to a unified software architecture. This allows users to store core data on flash storage and cold data on more cost-effective traditional disk storage. The data can freely mobilize between two types of storage systems according to preset policies, minimizing cost and management complexity.

As for hardware, Huawei launches chips supporting comprehensive capabilities pertaining to computing, storage, network, and management, to improve overall storage performance and reliability. Take a flash controller chip as an example. It is an essential component of an SSD. In the industry, only Huawei, Intel, and Samsung are capable of developing SSDs and SSD controller chips.

Through in-depth understanding of the underlying layer in a data center, we have designed proprietary high-performance, highly reliable SSDs. Competitors like Dell EMC and NetApp use SSDs from Toshiba, SanDisk, Micron, and Memblaze, which are powered by third-party controller chips. Huawei’s HSSD controller chips adopt hard logics to implement dynamic RAID, data inspection, and LDPC algorithms, providing higher efficiency and better performance than SSD controller chips of other vendors using built-in CPU algorithms.

Moore’s law, named after Gordon Moore, the co-founder of Fairchild Semiconductor and Intel, proposed the, is the observation that the number of transistors in a dense integrated circuit approximately doubles about every two years. It describes Intel’s success in crossing the industrial development gap and has accurately described the development of the IT industry for decades. Moore’s law not only describes technological development, but also represents a physical or natural law. Each successful product has its own “Moore’s law” describing its exponential growth trend. We must have an open mind and insist on learning and self-iteration to adapt to the changing times.

Putting User Experience First

It’s easier said than done when innovating services specific to users’ demands. User experience includes two aspects: Product experience (pertaining to product quality) and service experience (related to online upgrade, cloud, and maintenance services).

In terms of product experience, we envision of “Data on Demand,” aiming to build converged storage resource pools for cloud data centers, including all-flash storage ones, consolidate storage infrastructure, and remove differences of underlying storage products. This enables on-demand data migration and sharing between workloads by providing automated, on-demand, and consistent data services on- and off-premises. As a result, customers enjoy simpler, more agile cloud transformation.

Adhering to its vision, Huawei’s storage has been applied in many industries. In the big data cloud project of China Merchants Bank in 2017, Huawei’s storage helped the bank reduce TCO by 40% and speed up the resource provisioning tenfold compared with traditional storage. The rollout time of a new service was shortened from several weeks to several hours, facilitating efficient marketing decision-making.

In terms of service experience, Huawei’s storage products adopt artificial intelligence (AI) technologies and big data analysis to transform storage management, supporting intelligent device O&M and resource scheduling, and enabling data mining. This is one of the pillar fields of Huawei Storage in 2018. Huawei will release its intelligent storage management systems one after another. The soon-to-be-released eService is such an example. As an intelligent management system for remote access and control, eService will simplify storage device management and O&M and substantially increase overall O&M efficiency, cost-effectiveness, and reliability.

For example, faults may occur on running IT devices. To rapidly identify faults and recover from faults is a core reliability indicator. Huawei’s unique intelligent management chips serve as the management heart of storage. The chips have built-in fault diagnosis and pre-warning expert databases to improve fault diagnosis accuracy. Fast fault diagnosis is the prerequisite for fast data recovery. The chips provide 5x higher computing capabilities per second than peer products. Data switchovers can be completed within seconds in scenarios involving controller faults, front-end interface card faults, and management board faults. The chips ensure no data loss and user-unaware non-disruptive services during the switchover.

Intrinsically Motivated to March Forward

To date, Huawei’s storage products have witnessed an increasingly wider presence across the globe. The overall growth rate in sales revenue has long topped the industry. Remarkable achievements have been made in markets outside China, in particular: In 2017, the sales revenue of Huawei’s storage products increased by 147% in Europe and by 79% in the Asia-Pacific (excluding China). Meanwhile, Huawei continues to improve a stable sales pattern within China. In all-flash storage alone, the sales revenue grew at nearly twice the rate of 2017, topping in the industry.

Since its launch in 2011, Huawei OceanStor Dorado All-Flash Storage series has maintained a “zero accident” record. It has been trusted by key industry customers worldwide thanks to its proven product quality and technical capabilities. A large number of industry leaders in key industries around the world have chosen Huawei’s storage. Examples include Volkswagen, Vodafone, Saudi Aramco, Caixa Econômica Federal, Russian Post, Industrial and Commercial Bank of China, China Mobile, and China Pacific Insurance Group.

Many third-party analysts recognize Huawei’s strengths in storage. Huawei’s storage has been named a Leader in Gartner’s Magic Quadrant for General-Purpose Disk Arrays for two years in a row. American technical consulting firm DCIG ranked Huawei’s all-flash storage in the Recommended Ranking in the 2018-19 All-flash Array Buyer’s Guide, a higher rank than Dell EMC and HPE. In standard performance tests conducted by the Storage Performance Council (SPC) and Standard Performance Evaluation Corporation (SPEC), Huawei has always broken its own records. Among the Top 10 SPC-1 By Performance products, Huawei storage occupies six positions.

In today’s fast-paced world, the storage industry is undergoing profound changes accompanied by emerging hybrid clouds, intelligence, data lakes, storage media, data protection technologies, and new technical opportunities. Keeping up with Moore’s law and customer demands, Huawei will continue to release industry-leading products and technologies, such as faster SCM flash storage and software-defined distributed all-flash storage powered by end-to-end NVMe architecture and better proprietary storage chips. A 1 U all-flash storage system houses over 1 PB data. Huawei strives to become the data leader in the industry.

Internally, all product lines in Huawei strive to offer the best products. It is our constant pursuit to put customers first as we continue to innovate. Huawei’s storage products aim to top the world, offering unique and optimal customer experience.

We believe that our efforts and persistence allow Huawei’s storage product line to shoulder more responsibilities. Let’s embrace the future together.

The post Moore’s Law and Huawei Storage’s Continuous Improvement appeared first on Huawei Enterprise Blog.

Source: Huawei Enterprise Blog
—

Different Services Require Computing Resources of Vastly Different Scales 

Just like human beings having preferences for rooms and space in lives, so do enterprise service workloads. Workloads of different industries and businesses of varied sizes may have quite “personalized” requirements for IT deployment and computing resources. Even the workloads of the same enterprise may have different computing demands.

Huawei now presents the KunLun Mission Critical Server that is able to accommodate various enterprise databases and applications by offering the best-suited space and resources. Enterprises can plan the computing resources of KunLun based on their service requirements, and tailor the resources for databases and applications according to the most appropriate resource allocation ratio. This means that, in real-world deployment, enterprises can consolidate databases and applications into one KunLun server. The highly flexible resource dividing feature of KunLun helps customers save space, improve resource utilization, and maximize the return on investment.

One KunLun Offers Three Resource Dividing Modes 

KunLun is a high-end server built for mission-critical services. KunLun supports eight processors and more to deliver exceptional performance and reliability. KunLun supports three resource dividing modes: no partitioning (running as a single-node system), physical partitioning, and logical partitioning. 

In the no-partitioning mode, all computing resources of the entire server are used for a single type of workload. This is especially suitable for business-critical workloads demanding high performance and reliability.

In physical partitioning mode, one KunLun server is partitioned at a granularity of four physical processors. By doing so, KunLun offers highly isolated resources for different workloads with the desired flexibility. For example, the KunLun 9008 8S server (8S means 8-socket, that is, eight physical processors) can be divided into two 4-socket partitions, one for running databases and the other running applications.

The partitions are physically isolated. Any faults occurring on the processors, memory, and even power supply units (PSUs) of one partition do not affect the other partitions. This makes KunLun ideal for high availability (HA) deployment. For example, one KunLun 9016 (16-socket) server can be divided into four 4-socket physical partitions, all partitions running the Oracle database. An Oracle Real Application Cluster (RAC) can be deployed on these partitions. In the cluster, when a partition fails, the other three partitions will take over the services, ensuring zero downtime and zero data loss, and increasing database availability. This means, the SSD and HDD RAID recovery service provided by many services and companies can be avoided, as the data can be recovered internally.

The logical partitioning mode provides finer-grained resource division. Traditionally this technology used to be confined to high-end UNIX servers. As a high-end mission critical server, KunLun is the industry’s only x86 server that supports this feature. The logical partitioning technology allows the processor resources of KunLun to be divided on a minimum basis of one core. It addresses a particular scenario with enterprise IT systems: Some enterprises need to run multiple databases or applications at the same time. These workloads are associated with each other and need to be deployed as near as possible to improve performance. However, each database or application may require only a small amount of computing resources, generally just a few computing cores. Deploying each database or application on an independent server will leave a lot of resources idle and wasted on the servers.

In this case, the KunLun logical partitioning technology integrates the databases or applications into one server. The computing resources are divided and allocated to each database and application according to their actual needs. This ensures high resource utilization. Moreover, KunLun provides unified management software to simplify O&M. 

Consolidating Services Without Compromising Performance and Reliability

KunLun servers are more usually deployed in enterprise business-critical scenarios, which require KunLun to run services efficiently and stably after the databases and applications are consolidated. This in turn demands that KunLun be capable of isolating the physical resources as well as software and hardware faults among its physical partitions and logical partitions. These are the particular benefits brought to customers by KunLun.

Both in the physical and logical partitioning technologies of KunLun, each partition has independent computing resources, including computing cores, memory, and I/O ports. From the performance perspective, this avoids resource contention among different partitions, and maximizes performance for each partition at every moment. This is of vital significance to transactional services that have stringent real-time requirements. When a transaction request arrives at the server, if the corresponding partition does not respond promptly, the service experience may deteriorate or even the transaction will fail, which directly translates into operational loss. From the reliability perspective, this isolates faults between the partitions. Even if a fault occurs in a partition, the fault does not propagate to other partitions.

When a large number of logical partitions are deployed in a KunLun server, the number of I/O ports of a System Compute Enclosure (SCE) may not be sufficient to meet the needs of each logical partition. To address this requirement, KunLun supports Resource Expansion Enclosures (REEs), each REE providing up to 30 PCIe 3.0 slots to satisfy the I/O performance requirements of all logical partitions.

“Custom Space” Improves Return on IT Asset Investment

Whether running as a single-node system or processing consolidated services, KunLun excels. On the one hand, modern enterprises are tasked with swift digital transformation. To that end, they need to constantly expand and improve the IT infrastructure to match the rapid industry advance. On the other hand, enterprises seek intelligent methodology to improve the utilization of IT infrastructure and maximize return on their asset investment. As an integral part of the Huawei enterprise intelligent cloud infrastructure, the KunLun Mission Critical Servers differentiate themselves by offering ultimate flexibility and efficiency to meet enterprises’ demands for ever-higher resource efficiency. Meanwhile, KunLun unlocks leading-edge performance and reliability that empower enterprises to confidently grow critical services.

The post KunLun Offers Bespoke Space for Databases and Apps appeared first on Huawei Enterprise Blog.

Source: Huawei Enterprise Blog
—

By Wang Jiaxin from Huawei 

Speed is the ultimate weapon to survive and thrive in fierce competition. Today, NVMe is making the fastest speed possible to help you achieve your goals.

Designed for HDDs, traditional SAS protocol hinders SSD performance due to complex system architectures, excessive numbers of protocol to be parsed, and limited queue concurrences. NVM Express formulated NVMe protocol standards, and replaced complex protocol layers such as I/O Scheduler and SCSI in the SAS system with lightweight NVMe protocol. As a result, NVMe is a quicker, smarter, and more intuitive option for enterprises, evident with its superb performance in all-flash arrays (AFAs).

Huawei has extensively researched the NVMe protocols, covering only end-end development of NVMe flash controllers, NVMe all-flash OSs, and NVMe SSDs in the industry. Besides, powered by NVMe architecture with disk-controller collaboration, NVMe all-flash storage eventually ensures a stable latency of 0.5 ms.

 

NVMe-based AFAs outperform SAS-based AFAs, but the question remains: how?

First, at the transmission layer, SAS-based AFAs deliver I/Os from CPUs to SSDs through the following paths:

• Step 1: I/Os are transmitted from CPUs to SAS chips through the PCIe links and switches.
• Step 2: I/Os are converted into SAS packets before arriving at SSDs through the SAS switching chips.

For NVMe-based AFAs, I/Os are transmitted from CPUs to SSDs through the PCIe links and switches. CPUs of the NVMe-based AFAs directly communicate with NVMe SSDs over a shorter transmission path, resulting in higher transmission efficiency and less transmission latency.

Second, at the software protocol parsing layer, SAS- and NVMe-based AFAs differ greatly in protocol interaction technologies for data writes. A complete data write request requires 4 protocol interactions in SCSI protocol (connected through SAS back end); however, the NVMe protocol requires only 2 protocol interactions to complete a write.

Third, at the protocol encapsulation layer, when SAS protocol stacks are used, I/O requests are sent from block devices and reach SSDs through SAS links after the encapsulation of two layers (SCSI and SAS protocols). However, when the NVMe protocol stacks are used, I/O requests, however, require the encapsulation of only one layer (NVMe protocol). Simplified NVMe protocol stacks lower encapsulation costs by 50%, thereby reducing CPU consumption and I/O transfer latency caused by each encapsulation.

Fourth, at the multi-queue concurrency layer, the SAS protocol supports single queue, while the NVMe protocol supports up to 64 K queues, with each queue supporting a maximum of 64 K concurrent commands. Multi-queue NVMe protocol performs better thanks to higher concurrency processing and better cooperation with multiple channels and multiple dies in SSDs.

Fifth, at the lock mechanism layer, the single-queue SAS programming must be locked in the multi-core environment. Huawei designed an I/O scheduling mechanism for NVMe to completely cancel the disk-level mutex in original I/O paths and avoid I/O processing conflicts. This mechanism gives full play to the concurrent processing of multi-core processors, reduces software overheads, and improves back-end processing performance. In I/O scheduling, multiple threads work with multi-queues to achieve optimal performance.

Sixth, at the OS optimization layer, storage OSs designed for flash innovate the disk-controller coordination algorithms.

Be stable, be perfect 

Because of the advantages mentioned previously, customers choosing or creating new storage protocol are likely to be attracted to the many benefits of NVMe protocol. Despite offering superb performance unseen so far in the industry, the NVMe protocol has new challenges for disk and system design.

• Non-native dual-PCIe ports. Dual-controller redundancy technology is used in enterprise storage to ensure system reliability. However, PCIe-based dual-port technology is a challenge for SSD designs. Currently, some vendors’ SSDs do not use native dual-PCIe ports. For example, Intel’s NVMe SSDs use dual-port PCIe ports implemented through the internal PCIe switches. Such technology shortcuts inevitably reduce system reliability.
• Traditional PCIe hot plug with poor performance. Commercial NVMe SSDs must support non-disruptive replacement. However, traditional PCIe hot plug, particularly surprise hot plug, causes poor user experience due to system exceptions and service interruption.
• Data reliability guarantee pressure. As large-capacity disks are used, large amounts of data demand for higher reliability.
• Difficult cross-site reliability design. Reliability design for single points of storage has already been challenged. The cross-site reliability is going to be the same.

As one of the leading brands to successfully develop stable AFAs, Huawei continues to showcase its capabilities, evident in over 1000 NVMe all-flash success stories.

First, at the interface layer, Huawei NVMe SSDs support native dual-port technology, with two independent PCIe 3.0 x2 links. This provides hardware basis for system recovery and exceptions, and ensures dual-controller redundancy, helping improve the system reliability for enterprises.

Second, at the hot plug layer, Huawei is an expert in the telecommunication industry, especially regarding comprehensive PCIe link management, PCIe troubleshooting technology, and hot plug technology. The PCIe driver is designed to support SSD removal anytime and anyway, providing end-to-end PCIe system reliability if a single disk is replaced or a fault occurs.

Third, at the data protection layer, Huawei’s innovative RAID-TP software technology is based on the Erasure Code (EC) algorithm. Parity bits support 1-, 2-, 3-dimensions and can tolerate 1 to 3 simultaneous disk failures. This means that in the case of three disk failures, the system will not suffer from data loss or service interruption. Currently, only products from Huawei, NetApp, and Nimble can tolerate simultaneous failures of three disks; and other vendors (such as Dell EMC, HDS, and IBM) are unable to make this claim.

Although NetApp and Nimble can tolerate simultaneous failures of three disks, they both use traditional RAID architecture with fixed data disks and hot spare disks. For these companies, hot spare disk reconstruction for 1 TB of data takes 5 hours. OceanStor Dorado employs a global virtualization system able to reconstruct the data in a mere 30 minutes to fulfill requirements in ultra-large capacity profiles.

Fourth, at the cross-site data protection layer, Huawei NVMe all-flash storage provides comprehensive data protection technologies, such as snapshot, clone, and remote replication, to help customers build a hierarchical data protection solution from local or intra-city DCs to remote DCs. Huawei is a revolutionary member in implementing a gateway-free active-active solution in all-flash storage.

Get with the trend, prepare for the future

Traditional HDD storage has a latency of more than 10 ms due to the long seek time. However, SSDs reduce 50% of the storage system latency to about 5 ms by using electronic mapping tables.

Traditional storage controllers often provide the same OSs despite differences in HDDs disk form, meaning it is convenient to use even when the disk type changes. However, many of the HHDs and subsequent OSs have become redundant. That is why Huawei released AFAs, such as Pure Storage and OceanStor Dorado V3. Designed for SSDs, these AFAs effectively reduce the storage system latency to less than 1 ms.

In the future, faster storage media will undoubtedly be the next move for many enterprises looking to capitalize on innovative storage methods. Such is the benefits of using modern technologies. There is a large performance gap of 2 to 3 orders of magnitude between DRAMs and NAND SSDs, and even more between SCMs and DRAMs. All-flash storage using SCMs has the latency low to 250 μs, which ensures faster service response.

In addition to NVMe application in the local PCIe SSDs, NVM Express have also released the NVMe over Fabrics specification in June 2016. The new specification enables NVMe to be utilized over different fabric types, such as RDMA and FC, which can provide high-performance solutions for remote access to SSDs and remove resource sharing barriers among local SSDs.

Huawei uses NVMe over Fabrics to fully share SSD resources, and provides 32 Gbps FC and 100 GE full-IP networking design for front-end network connection, back-end disk enclosure connection, and scale-out controller interconnection. These functions decrease the storage latency and simplify storage network management by using one IP system to control the whole DC. This design avoids complex network protocol and planning, streamlines DC deployment, and reduces DC maintenance costs.

New SCM media are introduced to further improve system performance. With NVMe over Fabrics, SSD resources are fully shared, and front-end NVMe interfaces optimize hardware and software architectures. Ready to build the more competitive all-flash storage? Then think Huawei.

The post Lightning-Fast All-Flash Storage Ushers in an NVMe Era appeared first on Huawei Enterprise Blog.

Source: Huawei Enterprise Blog
—

By Wang Jiaxin from Huawei

In the digital economy era, data is growing explosively, transforming people’s understanding of data. Data used to serve business operations but has now become one of the driving forces enabling digital transformation for enterprises. As enterprises’ data centers carry an increasing number of businesses, data is used and mobilizes more and more frequently, requiring enterprises to find data systems that provide lower latency and higher service levels.

According to a survey on hundreds of data system users, about 87% of system performance problems occur in the interaction between the storage subsystem and the application database. That is to say, the response latency and concurrent access traffic of the storage subsystem determine those of the application system. High latency and small concurrent traffic of a storage subsystem has become the performance bottleneck of the entire system, which is an infuriating reality for many enterprises.

Let’s take a quick look at the history of latency requirements of storage systems needed for business expansion. In the HDD era, enterprise backup and web disk applications require relatively low storage latency, and 10 ms can help users easily cope with application demands. These days most virtual desktops require a latency of 5 ms, and with the emergence of cloud and virtualization technologies, virtual desktop offices have become the mainstream in large enterprises. Big data has changed business models, and the surge in data volume has a huge impact on enterprises’ customer relationship management (CRM) and enterprise resource planning (ERP) systems. The latency of 0.5 ms has become the ultimate goal of enterprises to ensure quality services.

The 0.5 ms latency is a predicted value that can operate under real-world heavy-workload conditions. It is neither a manipulated peak number produced in carefully controlled test parameters nor a number produced when systems are under zero workload. Storage systems are required to maintain 0.5 ms predictable latency even during service peaks so to provide users with a consistent experience.

Innovative FlashLink Technology

After continuously accumulating technical experience over the past 20 years, Huawei proudly launched the lightning-fast and rock-solid OceanStor Dorado V3 all flash storage in 2016. Still a player in the field today, it delivers the industry-leading performance powered by the innovative FlashLink technology, and the high performance is maintained from three aspects: chip, architecture, and operating system.

OceanStor Dorado V3 adopts three intelligent chips to achieve end-to-end service acceleration and provides a performance of 45% higher than SAS all-flash storage. Huawei is a groundbreaking telecommunication provider in that it continuously keeps up with the latest architectural technology trends and even develops its own technologies. For example, OceanStor Dorado V3 is one of the first all flash storage systems to use NVMe in commercial use. Further, OceanStor Dorado V3 adopts a brand-new SSD-optimized design and disk-controller collaboration technology to enable storage controllers to detect data layouts in SSDs in real time and synchronize data in controllers and SSDs. This helps reduce performance losses caused by garbage collection and ensures rapid response to data read and write I/Os. While these are just the highlights of OceanStor Dorado V3’s abundant back catalog, together they help maintain a predictable latency of 0.5 ms even under heavy workloads. The secret for such advancements is FlashLink, helping OceanStor Dorado to improve service performance by three times in comparison with traditional storage.

Innovative Disk-Controller Collaboration Ensures Predictable High Performance

The flash storage cells in an SSD can be re-written only after being erased. Generally, the basic writing unit of an SSD is a 16 KB page, and the basic erasing unit is an 8 MB block. To avoid erasing valid pages, valid pages in a block need to be migrated to another space for storage. The block space of valid pages is converted into invalid page space, after which the block can be erased at a time. The process of migrating valid pages is known as garbage collection.

Garbage collection improves the space re-utilization of an SSD, but each migration undermines the performance of the storage system. Large amounts of migrated valid data and shorter periods lasting from when each page is written to the SSD to when the page becomes invalid imposes greater impacts on the system performance.

To ensure that they are maximizing the performance of SSDs and flash storage systems, enterprises must effectively control garbage collection. Powered by proprietary SSDs and the flash operating system, OceanStor OS, Huawei’s OceanStor Dorado adopts an innovative disk-controller collaboration technology to prevent a drop in performance caused by garbage collection. By optimizing internal software algorithms, OceanStor Dorado enables storage controllers to detect the data layouts in SSDs in real time and make adjustments accordingly. This helps prevent data migration after being written to SSDs and garbage collection, ensuring predictable high performance for flash storage systems.

Large Block Sequential Writes Reduce the Frequency of Garbage Collection

Take real-time ridesharing as an example. Such services allow multiple users with similar routes to share a ride, which helps save travel fees of each user and reduce overall energy consumption.

Using this analogy, controllers of OceanStor Dorado V3 detect the data layouts in SSDs in real time and aggregate data blocks to be written to SSDs in the controller cache. The formats of the data blocks are unified, and then into a larger data block that is written to SSDs at a time to improve the overall system performance. Detailed benefits include:

• The large block sequential write technology controls the frequency at which random small blocks (I/O) are written to SSDs for multiple times, which makes full use of the bandwidth of back-end SAS.
• RAID write penalty (extra reads and writes required during verification) has long been one of the factors hindering the performance in a storage system using RAID protection. OceanStor Dorado V3 writes data into SSDs once after data aggregation, effectively reducing the number of disk writes and the number of extra read and write requests required for verification. This feature ensures a predictable system performance when RAID 5, RAID 6, and RAID-TP are used. Even in such an unlikely scenario as three disks failing concurrently, RAID-TP, a unique technology of OceanStor Dorado V3, can still ensure that services are unaffected to.
• OceanStor Dorado supports global garbage collection. It monitors the system pressure in real time and controls the frequency of garbage collection in disks, mitigating the impact of garbage collection on system performance. 

Independent Metadata Partition Controls the Frequency of Garbage Collection

In a storage system, the frequency of updating user data differs from that of updating metadata in that metadata is often updated more frequently. In scenarios where metadata and user data are written into the same partition on a disk, more garbage collections are required than in user data-only scenarios. This is because when pages of metadata become invalid, pages of the user data may remain valid. Therefore, a large amount of user data needs to be migrated during garbage collection, resulting in excessively large write amplification on the disk as well as shorter service life and lower performance of the SSD.

OceanStor Dorado V3 all flash storage uses independent metadata partitioning. It frequently writes updates metadata to a partition and infrequently updates to a different partition in the storage system and SSD. This reduces migration of user data blocks when upgrading metadata, mitigating the impact of garbage collection on system performance. In simple terms, the independent metadata partitioning technology controls the number of garbage collections, ensuring a predictable high performance of the storage system. 

Prioritizing Data Read and Write I/Os

Large financial enterprises, such as big banks, often set a special counter for VIP customers to separate them from the regular customers. Similarly, when VIP counters are busy, VIP customers can jump the queue to common counters. This kind of model ensures that VIP customers enjoy the most effective services at a quicker speed than regular customers.

This kind of model is also adopted by OceanStor Dorado, which introduces an I/O priority scheduling mechanism to ensure predictably low latency of service requests. OceanStor Dorado prioritizes data read/write requests with IT resources including CPUs, memories, and concurrent disk access traffic in storage systems. Other requests such as data reconstruction, asynchronous cache flushing, and background requests within the system should compromise in the case of resource contention.

The OceanStor Dorado priority adjustment is performed synchronously in the storage controller and the SSD to ensure that the data read/write requests enjoy the top priority at all times. Other types of data I/O requests are suspended when the read/write request arrives, and resume after the read/write operation completes, guaranteeing an optimal response latency of data read and write in the storage system. 

3x Higher Performance of All Flash Storage

In the digital transformation era, replacing traditional storage with all flash opens a new chapter. Individuals and enterprises no longer need to wait for response of applications in life and work.

For the financial industry, especially those in the securities field providing frequent real-time transactions, time is money. Take Hundsun in China as an example. Before it cooperated with Huawei, its traditional IT architecture supported only 60,000 transactions per second (TPS) at business peaks, insufficient of the desired 100,000 TPS. OceanStor Dorado V3 can helps enable Hundsun to process 150,000 transactions per second, and can scale for future business expansion.

In the manufacturing industry, batch processing capabilities of data warehouses are the basis for ERP business analysis. For example, BYD, the largest new energy vehicle manufacturer in China, needed at least 3.5 hours a day to batch process business requests. When the business volume was large, the system spent too long processing these requests, which caused great pressure on those at decision-making levels regarding service the next day. However once they implemented Huawei’s OceanStor Dorado V3, the system batch processing takes only 1 hour and 12 minutes, reserving sufficient time for final decision making.

In the medical industry, the hospital information system (HIS) is the core for hospital service management. It connects to multiple processes, such as registration, diagnosis, treatment, charging, and medication. Take a well-known tertiary hospital in China as an example. In its traditional IT architecture, each patient spent 3 seconds for registration on average and should wait in three to six queues during diagnosis, lasting for at least one hour. After Huawei OceanStor Dorado is used, the registration time of each patient takes only 0.5 seconds, improving diagnosis efficiency and improving the doctor-patient relationships.

Storage leasing is one of the main services of carriers and independent service providers (ISPs). Taking ACESI Group, the largest ISP in eastern France as an example. The speed of batch VM deployment was a major concern of ISPs because it was related to new business rollout. It took 30 minutes to deploy 100 VMs using traditional storage, making rapid service development impossible. OceanStor Dorado V3 shortened the deployment time of 100 VMs to just 10 minutes. In addition, Huawei enabled ACESI to develop new platinum leasing services based on the high-performance Dorado all flash storage, enhancing the overall competitiveness ACESI. This makes ACESI stay ahead of competitors in the industry.

In the future, Huawei’s OceanStor Dorado V3 all flash storage will benefit more customers.

The post FlashLink: The Secret to Huawei’s High-Performance All-Flash Storage appeared first on Huawei Enterprise Blog.

Source: Huawei Enterprise Blog
—

By Wang Jiaxin from Huawei

High-performance flash storage carries mission-critical business systems for various industries. If a problem occurred, enterprises would be hit hard. Qualix Group released figures to show the impacts of business interruption. In transportation, a one-minute stoppage would result in average losses of 150,000 USD, while that for banks would be 270,000 USD. The same one-minute stoppage for a telecommunications company would cost an average of 350,000 USD, while manufacturing would be hit with a 420,000 dollar loss, and security traders would top the list, losing 450,000 USD.

Therefore, ensuring mission-critical business continuity is a top priority for all-flash storage systems. Reliability, designed in an end-to-end way, is no easy task. Looking from media to systems and solutions, let’s examine how Huawei OceanStor Dorado all-flash storage can provide high performance and reliability for customers.

Disk-level reliability 

SSD reliability is measured by examining the mean time between failure (MTBF) and annualized failure rate (AFR). The industry MTBF benchmark is between 2 and 2.5 million hours. Huawei raises the bar well beyond this, reaching 3 million hours between failures on its homegrown disks.

How does Huawei accomplish this feat and extend the life of its SSDs? Huawei has maintained long-standing cooperation with its vendors, such as Samsung, Micron and Toshiba, to ensure that components are manufactured according to Huawei’s solution design objectives. Another reason is the extensive cooperation achieved between arrays and disks, which combines a series of reliability designs (such as optimization in graphene dissipation technology (GDT), global wear leveling, and global anti-wear leveling).

1. Global wear leveling design

At the beginning of the SSD lifecycle, service loads are spread in a balanced manner throughout SSDs to avoid overloading specific disks. This leads to the idleness of some disks and premature retirement of others.

2. Global anti-wear leveling design (Huawei-patented) 

At the end of the SSD lifecycle, when the wear on an SSD exceeds 80%, the anti-leveling mechanism takes the gradient wear to more than a 2% difference to avoid simultaneous disk failure. The service life of the system can be prolonged by gradually replacing the disks, ensuring sufficient time for system upgrade.

3. System software optimization 

First, at the algorithm layer, Huawei is the first vendor to commercialize the LDPC algorithm in SSDs. After years of optimization, Huawei now supports a 4 K ultra-long code algorithm. This brings the error correction granularity to twice of other SSD providers in the industry.

Second, at the flash chip layer, the number of erase cycles is limited in an SSD. The service life of an SSD can be prolonged if the number of erase cycles can be increased through algorithms. Huawei’s innovative adaptive program & erase (APE) technology automatically controls the erase strength and frequency of flash chips based on the amount of read and write data. In this way, the number of erase cycles can be effectively extended without changing costs or media granules, prolonging the SSD service life.

Third, at the data protection layer, while the storage controller system has RAID protection, SSDs also support two-dimension RAID groups with interleaving parity at channel and CE levels, ensuring chip-level failure data protection. The disk RAID and system RAID groups work together to conduct automatic data recovery if multiple chips of a single disk are faulty. Then, after being recovered, SSDs will be operational again.

System-level reliability 

Achieving reliability is complex. In addition to the hardware structure design and software fault tolerance mechanism, the storage system must tolerate physical and logical faults and support quick recovery. This will prevent data loss caused by system faults and ensure businesses continue running stably.

1. Magnitude 9.0 earthquake-resistant design 

The irregular seismic waves and intensified shaking caused by huge earthquakes will affect the stability and service life of electronic equipment. Huawei OceanStor Dorado all-flash storage has passed the magnitude 9 earthquake-resistant test run by China Telecommunication Technology Labs (TTL). This makes Huawei the only company to have done so and satisfy the TIL’s IT standards. Once an exception is detected, the system can also diagnose and rectify the fault quickly enough to prevent business interruption.

2. Tolerance of three-disk failures 

Disk capacity increases linearly with disk reconstruction time. Traditional RAID 5 or RAID 6 technologies allow 5 hours for the reconstruction of 1 TB of data, and 80 hours for 16 TB. However if one or two more disks become faulty during reconstruction, systems running RAID 5 or RAID 6 are unable to cope, severely disrupting business. Therefore, traditional RAID technologies cannot ensure system reliability, causing data loss and business interruption.

Huawei’s innovative RAID-TP software technology is based on the Erasure Code (EC) algorithm. Parity bits support 1-, 2-, 3-dimensions and can tolerate 1 to 3 simultaneous disk failures. This means that in the case of three disk failures, the system will not suffer from data loss or service interruption. Currently, only products from Huawei, NetApp, and Nimble can tolerate the simultaneous failure of three disks.

Although NetApp and Nimble can tolerate simultaneous failures of three disks, they both use traditional RAID architecture with fixed data disks and hot spare disks. For these companies, hot spare disk reconstruction for 1 TB of data takes 5 hours. OceanStor Dorado employs a global virtualization system able to reconstruct the data in just 30 minutes, fulfilling the requirements of ultra-large capacity profiles.

3. End-to-end data integrity protection and tolerance of silent data corruption 

In data access, any errors that occur can cause issues for data integrity when data is transferred through multiple components, channels, and complex software. However, such errors can only be detected in subsequent data checks and access. This phenomenon is called silent data corruption.

Often overlooked, silent data corruption has greatly impacted services, such as databases, that require absolute data integrity. Launched by Huawei, Emulex, and Oracle, the data integrity solution changes the traditional condition where hosts and storage systems protect data independently. This has been achieved by implementing end-end protection across applications, hosts, storage systems, and disks. As a result, this solution prevents silent data corruption for mission-critical businesses and eliminates potential down times.

4. Intelligent prefetch 

When a disk detects block faults or even severe die failures, the storage system receives failure reports from SSDs and uses redundant data in RAID groups to rapidly reconstruct and repair damaged data, reducing data loss risks and ensuring system reliability.

Huawei all-flash storage systems can accurately query internal data, such as SSD data, and use innovative prediction algorithms to monitor and predict the service life of disks. The personnel in charge of customer businesses will be told that their disks need replacing before the disks become faulty or one month before the service life is exhausted.

Solution-level reliability

Huawei OceanStor Dorado all-flash storage supports multiple data protection technologies, such as snapshot, clone, remote replication, and active-active data protection. This allows it to implement data protection solutions from local or intra-city to remote disaster recovery. This solution provides high availability and non-disruptive storage data services for customers, preventing data loss caused by logical or physical disasters.

1. Lossless snapshot 

Traditionally, COW-based snapshot technology requires data to be written to a location after being read and migrated to a new location. Therefore, such snapshot processes involve one read, two writes, and one metadata update. COW-based snapshot affects system performance due to performance loss during each data migration.

Huawei OceanStor Dorado all-flash storage implements lossless snapshot using ROW. When a snapshot is activated, data is written to the new location and the pointer of the mapping table is modified. Only one data write and one metadata update are involved, with data operation complexity being only 1/3 of that seen for COW-based snapshot. In addition, no extra data migration is required when the ROW snapshot is activated, resulting in no compromises in performance regarding production businesses.

In addition, OceanStor Dorado storage supports second-level periodic snapshot, which is superior to the minute- or hour-level snapshots used by competitors’ all-flash storage. OceanStor Dorado snapshot provides users with a more intensive and powerful continuous data management (CDM) solution, enabling real-time data protection.

2. Gateway-free active-active architecture 

Huawei OceanStor Dorado storage adopts a gateway-free active-active layout, removing the gateways on both sides. This immediately reduces customer procurement costs and lowers possible failures, achieving reduced latency, improved reliability, and accelerated performance. In addition, the overall networking is greatly simplified, with the number of deployment steps halved, thereby shortening the delivery cycle.

HyperMetro is deployed on two arrays in an active-active profile. Data on the active-active LUNs at both ends is synchronized in real time, and both ends process read and write I/Os from application servers to provide the servers with parallel active-active access. Should either array encounter a fault, services are seamlessly switched to the other end without interrupting service access, achieving RPO = 0 and RTO ≈ 0.

In remote data protection scenarios, the active-active solution can be effortlessly upgraded to the data center solution in geo-redundant mode, requiring no extra gateways and causing no business interruptions. This allows it to deliver a huge reliability protection rate of 99.9999% for customers. Third-party sites can even use Huawei OceanStor converged storage systems to provide cost-effective DR solutions for remote DR centers that require only ordinary response times.

Summary

All-flash storage focuses on performance and efficiency. Like a giant container ship on the sea, all-flash storage continuously pursues higher speeds and a larger capacity. With continuous and stable operations, Huawei OceanStor Dorado all-flash systems can deliver 99.9999% reliability, providing the public with a lightning-fast, rock-solid platform.

The post Huawei All-Flash Reliability Assurances for Your Mission-Critical Businesses appeared first on Huawei Enterprise Blog.

Source: Huawei Enterprise Blog
—

By Wang Jiaxin from Huawei

For most people, the first word they associate with all-flash system is fast. Storage vendors spare no effort to enhance the indicators of various flash systems, which has led to such developments as million-level IOPS, 0.3 ms, and even 0.1 ms latency. On face value, it would appear that SSDs can tremendously improve storage performance.

However, this is not the case. Not all products can fully offer all the benefits it claims to provide. For example, nearly all cars have the scale of 200 km/h on its dashboard, but few can actually operate freely when driving at this speed. Without solid chassis, precise steering, and good suspension, a car is taking risks when it runs at fast speeds. For storage, these components are all present in all-flash. The all-flash operating system can easily operate the all-flash super sports car (storage), making it run steadily under extreme road conditions.

IT equipment is configured with a large number of integrated circuits and PBC boards. The equipment design and production require sophisticated technologies and strict process control. Cutting-edge technology, sufficient capital, and decades of experience are all crucial for manufacturing reliable IT hardware devices. Storage is one of the most important aspects for all enterprises. It adopts the integrated design of hardware and software. All hardware needs to work with software to maximize its capability, which further enhances difficulties in storage engineering.

Hardware is the carrier of storage, and software is the soul of storage. Neither of them can survive independently. It requires great efforts to manufacture hardware, and even greater efforts for top-quality software. Manufacturing the operating system, considered the core of software, is the most demanding and time-consuming part. A common storage operating system contains at least 17 million lines of codes, requires the diligent work of 3000 first-class engineers, and at least five years of development and tests before it is completed. Over 65% of all development tests are made to test the fault-tolerance designs to ensure the high availability of the system. Only a handful of vendors in the industry have achieved such a feat. Most storage startup vendors’ are incompetent in designing a complete storage operating system. Therefore, a storage leader’s strength lies in an independent storage operating system.

With years of industry experience accumulated from varying industries, Huawei has developed its own state of the art storage operating system – OceanStor OS. For over eight years, this OS has been running on more than 50,000 sets of devices on the live network of enterprises, ensuring the stability and reliability of enterprises’ key data. By focusing on the most popular SSD technology in the industry, Huawei is one of the vendors holding a large number of SSD-level patents. OceanStor OS is based on a stable software platform and many SSD-level patents. Recently, Huawei launched a new OceanStor OS version dedicated to flash storage to fully utilize the capabilities of SSDs and flash systems.

Another aspect of Huawei’s cutting edge OceanStor products is the use of SSDs and garbage collection. For SSD read/write, SSD flash cells can be re-written only after being erased. Generally, SSDs write data by page (size: 16 KB) and erase data by block (size: 8 MB). Each block consists of multiple pages comprising valid pages, invalid pages, and empty pages. In an SSD, if pages within a logical location are newly written, the pages of original physical location become invalid. To avoid erasing valid pages, valid pages in a block must be migrated to another space. When all valid pages in the block are converted to invalid pages, the entire block can be erased at one time. The process of migrating valid data is known as garbage collection. Garbage collection improves the space re-utilization of an SSD, but each migration undermines the performance of the storage system. In addition, larger amounts of migrated valid data and shorter periods lasting from when each page is written to the SSD to when the page becomes invalid imposes greater impacts on the system performance.

Innovative disk-control collaboration lays a solid foundation for high performance

The key to effectively controlling garbage collection is to maximize the performance of SSDs and flash storage systems. Powered by proprietary SSDs and the flash operating system, OceanStor OS, Huawei’s OceanStor Dorado adopts an innovative disk-controller collaboration technology. By optimizing internal software algorithms, Dorado enables storage controllers to detect the data layouts in SSDs in real time and make adjustments accordingly. In this way, the data layout in the storage controller and SSDs can maintain consistent with each other. This prevents data migration and garbage collection after the data is written to SSDs, ensuring consistently high performance for flash storage systems.

Huawei-patented global wear/anti-wear leveling technology prolongs SSD service life

Different from HDDs, SSDs can only withstand a limited number of read and write operations. The service life of an SSD is an inverse proportion to the amount of data written to the SSD. Therefore, an all-flash storage system requires load balancing between multiple SSDs to prevent overly-used disks from failing.

OceanStor OS adopts a global wear leveling technology. Based on the collaboration between the controller software and SSD drives, all data is evenly distributed to multiple SSDs to share the service pressure. In addition, OceanStor OS periodically queries the SSD controller for the disk wear degree, and uses the wear degree as the basis for space allocation, thereby ensuring the reliability of the entire system.

However, when an SSD disk enters the end of its service life, for example, the disk wear degree reaches 80% or higher, multiple disks may be faulty at the same time, resulting in data loss. Huawei has developed a patented global anti-wear leveling technology to prevent SSDs from being faulty in batches.

OceanStor OS selects the most severely worn SSD and writes new data onto it as long as it has idle space. This reduces that SSD’s life faster and users are advised to replace it sooner, avoiding potential simultaneous failures and service interruptions. This technology is perfect for scenarios requiring IT devices replacement.

Purchasing a batch of new IT devices to replace the existing system is not an overnight task. It takes a long time to complete the procurement process approval, new device deployment, legacy service migration, and user acceptance. This is a very delicate process for users’ core services and data, so the effects are major if all SSDs break down during this period. Global anti-wear leveling technology can gradually replace the faulty SSDs and prolong the service life of legacy devices on the live network until official rollout of new devices.

Global inline deduplication and compression improve efficiency and service life SSD service life

Based on its analysis on HDDs and SSDs, IDC predicts that the price of an SSD will drop from more than 3 times in 2016 to 2 times at the end of 2018 over a 10K SAS disk. To accelerate the commercial use of SSDs, the industry uses inline deduplication and compression technologies to reduce the data volume before data is written into SSDs and minimize the amount of data that actually moves into SSDs, without affecting user experience.

Huawei OceanStor OS developed global inline deduplication and compression technologies. To obtain the best data reduction ratio, different types of services require different deduplication and compression granularities. The weak hash algorithm and byte-by-byte comparison are used for deduplication. After data is divided into data blocks by service type, deduplication is started. Then the system uses the weak hash algorithm to calculate the fingerprints of the data blocks and compares the fingerprints with the existing ones. If the fingerprint of a data block exists in the system, the system does not write the data block but only increases the fingerprint count. If a fingerprint is unique, the system adds it to the fingerprint table and writes the data block to SSDs. Deduplication is performed in real time, not after data has been written to SSDs.

The byte-by-byte comparison technology for addendum compares deduplicated data by byte to prevent the fingerprint hash conflict and ensure 100% data reliability.

The compression algorithm is a compute-intensive program. Inline compression consumes significant CPU resources, affecting end-to-end performance of the system. Industry peers often use the open source compression algorithm with high performance and low compression rate, such as LZ4, LZO, and Snappy. Huawei’s OceanStor OS is optimized based on the open-source LZ4 compression algorithm. The unit for storing compressed data is 1 KB, which doubles the compression efficiency and saves the storage space for compressed data.

Most all-flash storage vendors in the industry claim that their operating systems support inline deduplication and compression technologies. However, there are technological differences between OceanStor Dorado V3 all-flash storage and other products. According to the actual project test, under the data model of dual-controller, 100% random 8 KB I/O blocks and 7:3 mixed read/write, by stimulating the most common database scenarios, OceanStor Dorado with inline deduplication and compression enabled can maintain a 0.5 ms low latency and high performance. In the same test environment, the performance of OceanStor Dorado is twice that of EMC VMAX 950F or HPE StorServ 20850.

Based on the efficient deduplication and compression algorithms, Huawei promises a 3:1 data reduction ratio to customers who purchase OceanStor Dorado V3 all-flash storage series, helping users save investment and achieve higher return on investment (ROI). What’s more, if the guaranteed ratio is not met, Huawei is liable for providing additional storage capacity or exempting price on the capacity in future procurement. This function improves the storage system utilization, reduces user’s effective capacity cost per GB, maximizes space occupation, and reduces power consumption, lowers air conditioning and maintenance cost, helping to the end-to-end OPEX. Less data written into SSDs reduces the wear of SSDs, and prolongs the service life of SSDs and the storage system.

Comprehensive data protection and efficient software maximize the advantages of all-flash high performance

Thanks to years of expertise accumulation, Huawei’s new-generation OceanStor OS applies features of data protection software, including clone, remote replication, active-active, and 3DC, and inherits features from high efficiency software, including thin provisioning, QoS, and heterogeneous Virtualization. In addition, Huawei has fully optimized SSDs and developed more competitive features, such as lossless snapshot, RAID-TP that tolerates three-disk failure, and non-disruptive data migration, staying ahead of competitors in the all-flash era.

In addition to supporting gateway-free active-active mode and ensuring critical services with z zero RPO and a close-to-zero RTO, OceanStor OS is ahead of its peers in terms of performance, reliability, and efficiency.

• For performance, OceanStor OS is based on the high performance of all-flash storage and the optimization of internal lock mechanism in the active-active software, enabling Dorado all-flash storage to reach 200,000 IOPS at 1 ms latency, topping in the industry.
• For reliability, OceanStor OS supports upgrading an active-active solution to a geo-redundant solution to ensure 99.9999% high availability for critical services.
• For efficiency, OceanStor OS enables Huawei’s all-flash storage to support HyperMetro for both SAN and NAS. The integration of SAN and NAS changes the traditional active-active solution where extra gateways are added on arrays to provide active-active SAN and NAS services, decreasing the number of devices by more than two and reducing deployment complexity and costs. In addition, the advantages of SAN and NAS parallel architectures are fully utilized to improve service performance.

In addition, network integration changes the coexistence of multiple networks, such as Fibre Channel and IP networks. Between active-active sites, Fibre Channel or IP is used to deploy data replication networks, configure networks, and heartbeat networks in a unified manner, reducing deployment costs. In the traditional storage active-active solution, two arbitration mechanisms work separately, leading to inconsistent arbitration results of SAN and NAS services in the event a network fault occurs between sites. OceanStor OS adopts unified arbitration to ensure that in all instances SAN and NAS services are deployed at the same site and share the same number of resources.

Currently, one of the industry’s largest active-active clusters supports only eight nodes, and it cannot meet storage performance requirements in large-scale deployment scenarios. Huawei’s integrated SAN + NAS active-active solution inherits the scale-out architecture of common clusters and supports a maximum of 32 nodes in active-active mode, meeting customers’ fast-growing requirements for storage performance.

Yahoo, the Japan’s largest Internet company, uses Huawei’s integrated SAN+NAS all-flash active-active solution to ensure real-time synchronization of inventory data in online stores, maintaining consistency between online and offline inventory. This solution completes fault switchovers (180 km far between two sites) within seconds, five times faster than a solution provided by NetApp.

Manufacturing all-flash operating system is not an easy task. The preceding description of OceanStor OS is just the tip of the iceberg. When enterprise users actually put the complete set of OceanStor OS in use, they can find out the unique advantages by themselves. Ultimately Huawei is nothing without its customer base. Thankfully, after 20 years of successful deployment and intuitive research, Huawei has become of the world’s leading brands with a customer base ranging across multiple countries and industries, and has since become a brand that you can trust.

The post OceanStor OS Enables Fast and Stable All-flash Storage appeared first on Huawei Enterprise Blog.

Source: Huawei Enterprise Blog
—

By Qin Xuan from Huawei

In recent years, flash has become a hot topic. All-flash arrays are critical for enterprises to cope with explosive data growth and to accelerate mission-critical applications. According to data from IT Brand Pulse, the market share of enterprise-level HDDs has been declining since 2015. In 2017, the market share of SSDs exceeded that of traditional HDDs. It is expected that all-flash arrays will soon become mainstream data center storage and be used for key enterprise business.

Today, let’s talk about the performance of all-flash arrays. We all know that the all-flash arrays have high performance, and we often see storage vendors painstakingly promoting all-flash performance via mainstream media in the market. All-flash arrays can offer a performance ranging from hundreds of thousands of IOPS to millions of IOPS or even higher. Therefore, we must ask four questions to learn more about the performance of all-flash arrays.

What IOPS and latency can all-flash arrays provide?

Vendors must provide their IOPS and latency during promotion.

Generally, random read/write performance of an enterprise-class 10,000 rpm SAS disk offers about 200 to 300 IOPS. In comparison, the performance of a single SSD can reach thousands or even tens of thousands of IOPS, depending on the SSD type (SLC, MLC, or TLC). When the performance of an all-flash array exceeds a certain threshold, the latency increases accordingly. Therefore, when vendors claim that they can provide very high levels of performance, we must ask one question: how long is the latency? 0.5 ms? 1 ms? or 5 ms?

Huawei OceanStor Dorado V3 excels in both performance and latency. The following figure shows the performance data of OceanStor Dorado6000 V3, which was tested by the Storage Performance Council. The average latency is just 0.38 milliseconds, and when performance reaches the maximum IOPS of 1 million, its latency is less than 0.5 ms.

In addition, we must check whether the IOPS is stable or maximum performance.

What are application scenarios and workloads?

The performance of all-flash arrays varies with application scenarios, application models, and workloads. Workloads we are discussing here include:

• I/O size (8 KB, 32 KB, 64 KB, or 128 KB)
• Ratio of reads to writes (that is, the ratio of read I/Os or write I/Os in a host I/O request)
• Sequence and randomness of I/Os
• Other factors (such as cache hit ratio)

 

In this case, any change to a parameter (for example, 8 KB I/Os and 64 KB I/Os, 100% random reads/writes and sequential reads/writes, 7:3 mixed reads/writes, and 100% reads) can greatly affect performance. Therefore, when seeing very high levels of performance, you must ask the vendor what the system workloads like in order to achieve this performance.

In most real-world production environments, database OLTP is a typical application scenario. Therefore, when evaluating the performance of all-flash arrays, the commonly used workload models are 8 KB I/Os, 100% random I/Os, and read/write ratio 7: 3.

What is the performance of all-flash arrays when value-added functions such as deduplication and compression are enabled?

For all-flash arrays, inline deduplication and compression not only improve the service life of SSDs, but also help you reduce space and power consumption, further slashing costs. IDC reports show that data reduction functions such as inline deduplication and compression are key factors and technologies for reducing the costs of all-flash arrays.

To keep up with this trend, most traditional storage vendors have launched all-flash array models or versions. However, due to inherent weakness in the architecture, even though deduplication or compression is supported, all-flash array performance is seriously affected when deduplication, compression, or snapshots are enabled.

Take vendor H’s all-flash product as an example. The all-flash array is configured with two controllers, 48 x 480 GB SSDs, and RAID 10 groups, allowing the array to provide about 150,000 IOPS (workloads: random 8 KB I/Os, read/write ratio 7:3, and 1 ms latency). When RAID 5 groups are used, the array can only offer about 130,000 IOPS. However, after enabling the deduplication function, performance is halved, providing only 60,000 to 70,000 IOPS. If other value-added features (such as snapshots, replication, or active-active) are enabled, performance will see further massive decreases.

Therefore, we should focus on maintaining excellent performance while value-added functions are enabled. This is much closer to real-life application environments.

What is the long-term steady performance when capacity usage reaches 80%?

For example, at a customer’s POC site, vendor E’s all-flash arrays showed outstanding performance to begin with. However, as the amount of test data increased, performance deteriorated. The root cause of this was that the garbage collection mechanism started running in the background to handle increasing data volumes. The mechanism’s running mode and efficiency have a major impact on all-flash array performance.

A simple way to establish or simulate a real-life application scenario is to embed around 80% of data (or perform random overwrites for one time) before starting an official performance test. By doing this, you can easily test the long-term steady performance of all-flash arrays.

The OceanStor Dorado V3 uses the global garbage collection mechanism, which is triggered as a background task when the ratio of idle blocks reaches a specified threshold. The OceanStor Dorado V3 uses system-level end-to-end I/O priority to ensure that I/Os generated by garbage collection have the lowest possible impact on host I/O performance, thereby ensuring the stable latency of host I/Os.

• On-demand start and stop of garbage collection: Controls matching between garbage collection bandwidth and front-end write I/O bandwidth, migrates data on demand, and reduces write amplification, minimizing the impact of garbage collection on host performance.
• Optimized cost-benefit policy used to select target data for garbage collection: Separately stores cold and hot data (separation of metadata and data, frequently accessed data blocks and seldom accessed data blocks), moves invalid data to idle blocks based on the cost of releasing space, and batch reclaims data written at the same point in time. This reduces unnecessary data migrations and minimizes the impact of garbage collection on application performance.

The following figure shows a performance comparison between Huawei OceanStor Dorado5000 V3 and other mainstream all-flash arrays when under strict test conditions (two controllers, random 8 KB I/Os, read/write ratio 7:3, 80% occupation rate, inline deduplication and compression enabled, and 0.5 ms latency). We can see that Huawei’s product provides superior performance compared to the other storage arrays.

Summary

Equipped with native, flash-architecture design capabilities, Huawei is the only vendor that develops SSD chips, SSDs, and all-flash systems. Huawei enables disk and system collaboration to make the most of all-flash features, and Huawei all-flash arrays deliver the highest performance and reliability in the industry by using operating systems specially designed for all-flash arrays, unique FlashLink technology, and dedicated performance acceleration chips.

We hope that this article helps you to look at dazzling promotions and understand the true performance capabilities of all-flash arrays.

Huawei has demonstrated its amazing performance in multiple all-flash competition tests. Thanks to its strong product competitiveness, Huawei’s all-flash arrays are growing rapidly. In the first three quarters of 2017, Huawei’s all-flash storage revenue enjoyed the fastest growth speed in the world, and started to serve many famous enterprises worldwide such as Vodafone, Saudi Arabia Aramco, Russian Post, Brazil’s Caixa, and China Pacific Insurance.

The post Four Questions to Understand All-Flash Performance appeared first on Huawei Enterprise Blog.

Source: Huawei Enterprise Blog
—