Is RAID Dead?- Part 2
Is RAID Dead?- Part 1
Is RAID Dead?- Part 3
Is RAID Dead? - Part 2
The Emergence of Replica & Erasure Coding – Software Defined Storage
This Insight article is Part 2 of a 3-part series “Is RAID Dead”. If you have missed part 1, you may refer to our earlier post. We will now review a prima on Replica and Erasure Coding. This will lead on to part 3, a posit that RAID is at the end of its useful life.
RAID (Redundant Array of Independent Disks) was first invented in 1987 and still in use till today. It was designed to create a large storage system, made up of smaller HDDs (Hard-Disk Drives), to provide the capacity and to automatically recover and rebuild data around failed drives.
It is a proven system, otherwise it would not have lasted so long. However, newer storage systems, based on SDS (Software-Defined Storage), have eschewed RAID in favour of either Replica or Erasure Coding in building storage arrays.
Primer on Replica
Replica is making multiple copies of the data and storing it in different drives. SDS defaults to Replica 3, making 3 copies of every data when writing to drives.
Replica 3 can suffer up to 2 failed drives, provided that these 2 failed drives hold the data to be read. If the failed drives do not hold more than 2 copies of the data, then more drives can fail without interruption to operations.
The diagram below illustrates Replica 3.
3 copies of the photo are made, then stored into drives D1, B4, D7. Any 2 drive of D1, B4, D7 can fail without data loss.
As soon as any drive fails, the SDS system will detect it, and immediately commence data recovery. A replica of the data in that failed drive is made from other available drives, and stored in the unused space of other working drives. Over time, the SDS system will settle into a stable state, with full Replica 3 restored.
Failed drives can be replaced any time. Once a failed drive is replaced (or a new drive added), the SDS system will commence to balance all drives, by moving the Replicas around, so that all active drives will have the same average utilisation.
The total useable storage space for Replica 3 is N/3 x C, where N is the number of drives, and C is the capacity of each drive.
Primer on Erasure Coding
In Erasure Coding, data is sent to a mathematical function to break the data into smaller and equal sized shards, similar to RAID 5/6. The number of shards can be defined, eg 3 + 2 (3 data shards and 2 recovery shards). Other possible definitions are 3 + 3, 4 + 2, 5 + 2, etc.
The diagram below illustrates 3 + 2 Erasure Coding :
The photo is first broken up into shards D1, D2, D3, R1, R2 and stored into different drives in the storage array. To read back the photo, we only need any 3 of the 5 shards, and in the example above D1, D2, D3. In the illustration below, a drive containing D2 fails.
The same photo can be read back by D1 + D3 + R1, or D1 + D3 + R2.
In the 3 + 2 Erasure Coding illustration above, up to 2 drives that hold the shards can fail, without data loss.
As soon as any drive fails, the SDS system will detect it, and immediately commence data recovery. The photo is first read back from the available shards, then re-sharding will happen to get back D2. D2 is then stored in the unused space of other available drives. Over time, the SDS system will settle into a stable state, with full 3 + 2 Erasure Coding restored.
Failed drives can be replaced any time. Once a failed drive is replaced (or a new drive added), the SDS system will commence to balance all drives, by moving shards around, so that all active drives will have the same average utilisation.
The total useable space for a 3 + 2 (K = 3, M = 2) Erasure Coding is K/(K + M) x N x C, where K and M is the definition for the Erasure Coding, N is the number of drives, and C is the capacity of each drive.
The End of RAID
In the final installment of this Insight article, we will discuss about the advantages/disadvantages of RAID vs Replica and Erasure Coding. We will also posit that RAID is no longer useful as a storage technology, given the immense rise of the amount of data, and the advances in storage capacities of HDDs.
RELATED ARTICLES
Is RAID Dead? Part 1
Is RAID Dead? Part 2
Is RAID Dead? Part 3