Erasure Coding Scheme for Data Protection and Recovery
Describes the erasure coding (EC) schemes for data protection and recovery.
Erasure coding (EC) is a data protection method in which data is broken into fragments, expanded and encoded with redundant data pieces, and stored across a set of different nodes or storage media.
EC ensures that if data becomes corrupted, it can be reconstructed using other data and parity fragments.
The time required to reconstruct data depends on the number of data fragments in the chosen EC scheme, and the number of failures that have occurred. For example, reconstruction of EC scheme 10+2 takes longer compared to the reconstruction of EC scheme 3+2, as a larger number of data and parity fragments must be read.
Considerations When Selecting an EC scheme
As an administrator, consider the following points when selecting an EC scheme:
- How many nodes can you afford to have?
- How many failures might occur? Do you anticipate a single failure, or multiple failures?
- Consider the following when determining how long you are willing to wait for a node to
be rebuilt:
- Rebuilds overhead - For 4+2, need to read data from 4 nodes to rebuild, for 6+3, need to read from 6 nodes to rebuild.
- Reads overhead - For 4+2, need to read data from 4 nodes , for 6+3, need to read from 6 nodes. For degraded states, parity calculation overhead is an add-on.
- Data classification - High ecschemes 10+2 , 12+4, are ideal for archived data, since data mostly remains untouched.
EC Schemes
In an erasure coded volume, an erasure coding scheme has the stripe layout m+n. The stripe is an array of m data fragments and n parity fragments.
Each fragment is called a stripelet. Each stripelet is present on a container and one stripe is across different containers on different nodes. The default stripelet size is 4MB. For an EC scheme 4+2 for example, the stripe size is 24MB.
A Container Group (CG) is collection of such stripes. Based on the maximum size of a container (32 GB), the maximum number of stripes in a CG is 8K.
Each stripe is created by the same number of data fragments from all containers in the group of EC containers. Each container is placed on a different physical node.
- m is the number of data fragments.
- n is the number of redundant fragments (referred to as parity fragments).
- The parity is calculated using data from all data fragments.
- m/(m+n) is the encoding rate.
- m+n is the number of encoded fragments.
- You need to read a minimum of m blocks to recover data.
- You can recover data from a maximum of n failures.
For example, assume m=4, n=2, and stripe depth=4 MB.
- The number of data fragments is four (4), and while the number of parity fragments is two (2).
- The number of encoded fragments is six (6).
- The stripe size is 16 MB (4x4 MB) of user data, and 8 MB (2x4 MB) of parity fragments.
- The system can handle two (2) failures, and any fragment can be recovered from four (4) other fragments.
- The number of data fragments must be between 2 and 10 (both inclusive) for erasure coding scheme without local parity.
- The number of nodes must be greater than or equal to the sum of data and parity fragments.
- The number of parity fragments must be greater than or equal to 1 and less than or equal to the number of data fragments.
Select from the following schemes for erasure-coded volumes:
| EC Scheme | Number of Data Nodes | Number of Parity Nodes | Total Number of Nodes Needed | Number of Failures Recoverable | Number of Nodes to Read to Recover Data |
|---|---|---|---|---|---|
3+2 |
3 | 2 | 5 | 2 | 3 |
4+2 |
4 | 2 | 6 | 2 | 4 |
5+2 |
5 | 2 | 7 | 2 | 5 |
6+3 |
6 | 3 | 9 | 3 | 6 |
10+<x> where x is a value from 1 to 10 |
10 | x | 10+x | x | 10 |
Although you can create a volume without the required number of nodes for a specific scheme, volume offload fails if the required number of nodes are not present.
When choosing the scheme, note that more nodes leads to longer recovery time, resulting in degraded performance, network expense, and lengthy time to rebuild.For example, consider a 12 + 4 EC scheme
represented as D0 + D1 + D2 +....+D10 + D11 + P0 +…+P3
Suppose node D4 goes down, now to rebuild, a total of 12 stripelets must be read. This leads to huge performance degradation in network bandwidth, CPU cycles, and Disk IO .
To reduce the reconstruction cost, use EC Local Parity, where the number of stripelets to be read reduces to 6 for a single failure in the 12+2+2 scheme.