META CHUNKS

• US 20190317859A1
• Filed: 04/12/2018
• Published: 10/17/2019
• Est. Priority Date: 04/12/2018
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Danilov, Mikhail, Gusakov, Kirill
• Granted Patent: US 10,817,374 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/154 Error and erasure correction, e.g. by using the error and erasure locator or Forney polynomial

Abstract

Data protection with meta chunks increases capacity use efficiency without verification and data copying. In one aspect, a meta chunk is a data protection unit, which combines two or more source chunks that are determined to have a reduced sets of data fragments. The meta chunk can be encoded to generate a set of coding fragments, which can be stored and utilized to recover data fragments of any of the two or more source chunks. Further, the source chunks can be linked to the meta chunk. Furthermore, the sets of coding fragments, that were previously generated by individually encoding each source chunk, can be deleted.

Claims

1. A system, comprising:

a processor; and

a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising;

determining source chunks stored within a storage system, wherein the source chunks are determined to have fewer than a defined number of data fragments;

based on combining the source chunks, generating a meta chunk; and

encoding the meta chunk to generate coding fragments that are employable to recover at least a portion of the source chunks.

2. The system of claim 1, wherein the operations further comprise:

linking the source chunks to the meta chunk.

3. The system of claim 2, wherein the linking comprises updating metadata associated with a source chunk of the source chunks to comprise reference data indicative of the meta chunk.

4. The system of claim 1, wherein the coding fragments are meta chunk coding fragments and the operations further comprise:

subsequent to the encoding, deleting source chunk coding fragments that have been generated based on individually encoding the source chunks.

5. The system of claim 1, wherein the meta chunk is a first meta chunk, the source chunks are first source chunks, the first source chunks comprise a second meta chunk that has been generated based on a combination of second source chunks, and wherein the second source chunks are determined to have fewer than the defined number of data fragments.

6. The system of claim 5, wherein the operations further comprise:

subsequent to the encoding, deleting the second meta chunk.

7. The system of claim 1, wherein the encoding comprises erasure coding the meta chunk.

8. The system of claim 7, wherein the combined data fragments of the source chunks are determined to satisfy a defined criterion associated with an erasure coding protocol.

9. The system of claim 1, wherein physical capacity is not allocated for the meta chunk.

10. The system of claim 1, wherein the storage system comprises a geographically-distributed object storage system.

11. A method, comprising:

selecting, by a system comprising a processor, source chunks from chunks of an object storage system, wherein the source chunks are determined to have fewer data fragments than remaining of the chunks other than the source chunks;

combining the source chunks to generate a meta chunk; and

based on erasure coding the meta chunk, determining coding fragments for the source chunks at a meta chunk level, wherein the coding fragments are to be employed to recover at least a portion of the source chunks during a failure condition.

12. The method of claim 11, further comprising:

storing reference data that links the source chunks to the meta chunk.

13. The method of claim 12, further comprising:

in response to determining that the failure condition is satisfied, detecting that at least the portion of the source chunks is unavailable.

14. The method of claim 13, further comprising:

based on the reference data, determining the meta chunk linked to at least the portion of the source chunks.

15. The method of claim 14, further comprising:

recovering at least the portion of the source chunks based on performing a decoding operation at the meta chunk level.

16. The method of claim 11, wherein the coding fragments are first coding fragments, and the method further comprises:

subsequent to the determining, deleting source second fragments that have been generated based on individually erasure coding the source chunks.

17. A computer-readable storage medium comprising instructions that, in response to execution, cause a server device comprising a processor to perform operations, comprising:

combining source chunks stored within an object storage system to generate a meta chunk, wherein the source chunks are determined not to have more than a defined number of data fragments; and

based on erasure coding the meta chunk, determining coding fragments for the source chunks at a meta chunk level, wherein the coding fragments are to be employed to recover at least a portion of the source chunks during a failure condition.

18. The computer-readable storage medium of claim 17, wherein the coding fragments are meta chunk coding fragments, and the operations further comprise:

subsequent to the determining, deleting source chunk coding fragments that have been generated based on individually erasure coding the source chunks.

19. The computer-readable storage medium of claim 17, wherein the operations further comprise:

determining reference data that links the source chunks to the meta chunk.

20. The computer-readable storage medium of claim 19, wherein the operations further comprise:

updating metadata associated with the source chunks the reference data.