Data object recovery for storage systems

• US 10,152,376 B2
• Filed: 01/05/2017
• Issued: 12/11/2018
• Est. Priority Date: 06/29/2016
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Danilov, Mikhail, Buinov, Konstantin, Fomin, Andrey, Kurilov, Andrey, Trusov, Maxim
• Primary Examiner(s): Chaudry, Mujtaba M
• Application Number:
  US15/398,826
• Publication Number:
  US 20180004600A1
• Time in Patent Office: 705 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes

Abstract

A method comprising: receiving an I/O request for object data; determining one or more data fragments wherein the object data is stored; determining that one or more of the data fragments are unavailable; determining, from within the one or more unavailable data fragments, a set of slices storing the object data, each slice comprising k small data fragments and m coded fragments; for each slice, retrieving at least k small data and coded fragments within the slice from storage; and recovering a segment of the object data using the retrieved small data and coded fragments. A related system and computer program product are also described.

Claims

1. A method for retrieving data in a storage system, the method comprising:

receiving, at the storage system, an I/O request for object data, the I/O request being received from a client device;

identifying one or more data fragments where the object data is stored;

determining, by the storage system, that a first one of the data fragments is unavailable due to a failure of a node in the storage system, the first data fragment containing a plurality of first small data fragments that constitute a first portion of the object data;

retrieving, a plurality of second small data fragments and a plurality of coded fragments, the plurality of second small data fragments being generated based on the plurality of second small data fragments and a coding matrix, the plurality of second small data fragments and the plurality of coded fragments being part of a slice that is associated with the first data fragment;

recovering, by the storage system, the first portion of the object data by using the plurality of second small data fragments and the plurality of coded fragments; and

completing, by the storage system, the I/O request for object data based on the recovered first portion of the object data.

2. The method of claim 1, further comprising:

determining, that a second one of the data fragments is available;

retrieving the second data fragment; and

extracting a second portion of the object data from the second data fragment,

wherein the I/O request is completed further based on the extracted second portion of the object data.

3. The method of claim 1, wherein recovering the first portion of the object data comprises performing erasure decoding on the slice.

4. The method of claim 1, wherein the plurality of second small data fragments is retrieved from two or more different nodes within the storage system.

5. The method of claim 1, wherein each of the data fragments is greater than 100 megabytes and each of the first small data fragments is less than 1 kilobyte.

6. The method of claim 5, wherein each of the first small data fragments is a half-byte.

7. A storage system comprising:

one or more processors;

a volatile memory; and

a non-volatile memory storing computer program code that when executed on the processor causes execution across the one or more processors of a process for retrieving data comprising the operations of;

receiving an I/O request for object data, the I/O request being received from a client device;

identifying one or more data fragments where the object data is stored;

determining that a first one of the data fragments is unavailable due to a failure of a node in the storage system, the first data fragment containing a plurality of first small data fragments that constitute a first portion of the object data;

retrieving a plurality of second small data fragments and a plurality of coded fragments, the plurality of second small data fragments being generated based on the plurality of second small data fragments and a coding matrix, the plurality of second small data fragments and the plurality of coded fragments being part of a slice that is associated with the first data fragment;

recovering the first portion of the object data by using the plurality of second small data fragments and the plurality of coded fragments; and

completing the I/O request for object data based on the recovered first portion of the object data.

8. The storage system of claim 7, wherein the computer program code causes execution across the one or more processors of the process further operable to perform the operations of:

determining that a second one of the data fragments is available;

retrieving the second data fragment; and

extracting a second portion of the object data from the retrieved second data fragment,

wherein the I/O request is completed further based on the extracted second portion of the object data.

9. The storage system of claim 7, wherein recovering the first portion of the object data comprises performing erasure decoding on the slice.

10. The storage system of claim 7, wherein each of the data fragments is greater than 100 megabytes and each of the first small data fragments is less than 1 kilobyte.

11. The storage system of claim 10, wherein each of the first small data fragments is a half-byte.

12. A computer program product tangibly embodied in a non-transitory computer-readable medium, the computer-readable medium storing program instructions which when executed by at least one processor that is part of a storage system cause the at least one processor to:

receive an I/O request for object data that is transmitted to the storage system by a client device;

identify one or more data fragments where the object data is stored;

determine that a first one of the data fragments is unavailable due to a failure of a node in the storage system, the first data fragment containing a plurality of first small data fragments that constitute a first portion of the object data;

retrieving, a plurality of second small data fragments and a plurality of coded fragments, the plurality of second small data fragments being generated based on the plurality of second small data fragments and a coding matrix, the plurality of second small data fragments and the plurality of coded fragments being part of a slice that is associated with the first data fragment;

recover the first portion of the object data by using the plurality of second small data fragments and the plurality of coded fragments; and

complete the I/O request for object data based on the recovered first portion of the object data.

13. The computer program product of claim 12, wherein the program instructions are further executable to:

determine that a second one of the data fragments is available;

retrieve the second data fragment; and

extract a second portion of the object data from the retrieved second data fragment,

wherein the I/O request is completed further based on the extracted second portion of the object data.

14. The computer program product of claim 12 wherein recovering the first portion of the object data comprises performing erasure decoding on the retrieved slices.

15. The computer program product of claim 12, wherein the plurality of second small data fragments is retrieved from two or more different nodes within the storage system.

16. The computer program product of claim 12, wherein each of the data fragments is greater than 100 megabytes and each of the first small data fragments is less than 1 kilobyte.

17. The computer program product of claim 16, wherein each of the first small data fragments is a half-byte.