Method and apparatus for reconstructing data in object-based storage arrays

• US 20060156059A1
• Filed: 01/13/2005
• Published: 07/13/2006
• Est. Priority Date: 01/13/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Hitachi, Ltd.
• Inventors: Kitamura, Manabu
• Granted Patent: US 7,386,758 B2
• Time in Patent Office: Days
• US Class Current: 714/6
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 2211/1035 Keeping track, i.e. keeping track of data and parity changes

Abstract

A method and apparatus for placing objects on a storage device of a storage system and reconstructing data of objects in the storage device. The storage system stores data as objects and implements a RAID architecture including a plurality of the storage devices, and a disk controller for processing Object-based Storage Device (OSD) commands. Each object includes data and attribute. Parity data is calculated for reconstructing an object upon occurrence of a storage device failure. Each storage device includes plural stripes each having a predetermined length. Each object is stored in a stripe wherein an attribute is stored in the head of the stripe and data is stored after the attribute. When the object size exceeds the stripe length, the remainder of the object is stored in the next stripe, and when another object is to be stored, an attribute is stored at a head of a further next stripe and data is stored just after the attribute.

Claims

1. A storage system for storing data as objects comprising:

a plurality of storage devices; and

a disk controller for controlling said storage devices,

wherein each of said objects includes attribute which defines attributes of the object and data,

wherein said storage devices are configured to implement a Redundant Array of Inexpensive Disks (RAID) architecture,

wherein each of said storage devices is formatted to include a plurality of stripes each having a predetermined length and at least one logical unit (LU) is formed by all or a portion of one or a plurality of said storage devices,

wherein each object is stored on said LU such that an attribute of the object is stored in the head of a stripe and data of the object is stored just after the attribute,

wherein when another object is to be stored, an attribute of the another object is stored at a head of the next stripe and data of the another object is stored just after the attribute in the next stripe, said stripe which includes attribute and data or data is referred to as a data stripe, and

wherein according to the RAID architecture being implemented, parity data is calculated based on certain ones of the data stripes, said parity data is stored in a stripe of a storage device and such stripe is referred as a parity stripe.

2. The storage system according to claim 1, wherein the RAID architecture being implemented is defined according to a RAID configuration table which includes a plurality of entries each including a port identifier (ID), a logical unit number (LUN), a storage device (disk) ID, a RAID level, and a stripe size, thereby defining for each combination of port ID and LUN, a logical storage device which is formed by all or a portion of one or a plurality of said storage devices and a RAID level and stripe size to be used on said logical storage device.

3. The storage system according to claim 1, wherein said storage system responds to an object-based storage device (OSD) input/output (I/O) command, and

wherein said OSD I/O command causes said storage system to perform an 1/O operation on said object by specifying said logical unit according to a combination of a port identifier (ID) which identifies a particular port through which said logical storage device can be accessed and logical unit number (LUN) which identifies a particular logical unit where said object is stored.

4. The storage system according to claim 1, wherein status of each of said storage devices is monitored by a disk status table which includes a plurality of entries each including a disk ID which identifies a particular storage device, status information and spare disk information, thereby defining for each storage device whether said particular storage device has failed and a hot-standby storage device to be used as a replacement storage device if said particular storage device has failed.

5. The storage system according to claim 4, wherein said status information indicates whether said storage device is operating normally, has failed, or has been replaced by said hot-standby storage device.

6. The storage system according to claim 4, wherein said spare disk information indicates one of the other storage devices to be used as said hot-standby storage device to replace said storage device when said storage device fails.

7. The storage system according to claim 1, wherein a logical-physical (LP) mapping table is used by said storage system to manage relations between objects and stripes, and

wherein said LP mapping table is used to search for a location of an object when said storage system receives an I/O command.

8. The storage system according to claim 7, wherein said LP mapping table includes a plurality of entries each having a port identifier (ID) which identifies a particular port through which a logical storage device can be accessed, a logical unit number (LUN) which identifies a particular logical unit (LU) where an object is stored, a partition ID which identifies a partition on a storage device which forms the LU within which the object is stored, an object ID which identifies the object, and disk location information which identifies in which stripe the object, specified by the partition ID and object ID, is stored.

9. The storage system according to claim 1, wherein a physical-logical (PL) mapping table is used by said storage system to manage relations between disk blocks of each of said storage systems and objects, and wherein said PL mapping table is used to search which disk block contains a particular object.

10. The storage system according to claim 9, wherein said PL mapping table includes a plurality of entries each having a stripe identifier (ID) which identifies a particular stripe which contains a particular object, a port ID which identifies a particular port through which a logical storage device can be accessed, a logical unit number (LUN) which identifies a particular logical unit (LU) where the object is stored, a partition ID which identifies a partition on a storage device which forms the LU within which the object is stored, and an object ID which identifies the object.

11. The storage system according to claim 3, wherein said OSD I/O command is one of CREATE, WRITE, READ, CREATE AND WRITE, APPEND, GET ATTRIBUTE, and REMOVE commands, wherein said CREATE command is used to allocate one or more objects in the logical unit,

wherein said WRITE command is used for writing a certain length of data from a specified starting byte address in an object specified by an object ID,

wherein said READ command is used for reading a certain length of data from a specified starting byte address in an object specified with an object ID,

wherein said CREATE AND WRITE command is a compound operation which is used to execute a CREATE operation and a WRITE operation in a single command,

wherein said APPEND command is used to append a specified length of data after a current object,

wherein said GET ATTRIBUTE command is used for retrieving an attribute of an object for use in later processing of the object, and

wherein said REMOVE command is used for deleting an object in a logical unit.

12. The storage system according to claim 1, wherein when one of said storage devices fails, a data reconstruction process is implemented such that data recovery is performed on each data stripe where an object or a part of said object is stored.

13. The storage system according to claim 12, wherein when a storage device fails, said storage system creates for said storage device a list of stripes included in said storage device within which objects are stored.

14. The storage system according to claim 13, wherein said list is a disk reconstruction bitmap which includes a plurality of entries each including a stripe identifier (ID) which identifies a particular stripe, recovery information which indicates whether the identified stripe is to be recovered, and done information which indicates whether the data reconstruction of the identified stripe has been completed.

15. The storage system according to claim 13, wherein when a storage device fails said storage system assigns a replacement storage device to replace the failed storage device, starts a process to create said list, and reconstructs data according to said list using a RAID technique.

16. The storage system according to claim 1, wherein when a storage device fails, a data reconstruction process is implemented such that data recovery is performed on each data stripe where an object or a part of said object is stored, and

wherein during said data reconstruction process isolated stripes each being stored at locations on a storage device separated from other stripes are relocated to be adjacent to the other stripes.

17. The storage system according to claim 16, wherein when a storage device fails, said storage system creates for said storage device a list of stripes including only those stripes within which an object or a part of said object is stored and which are to be reconstructed.

18. The storage system according to claim 17, wherein said list is a disk reconstruction bitmap which includes a plurality of entries each including a stripe identifier (ID) which identifies a particular stripe, new location information which indicates a new location for the reconstructed data of the identified stripe, done information which indicates whether the data reconstruction of the identified stripe has been completed, and parity reconstruction information which indicates when whether a parity stripe corresponding to the identified stripe has been reconstructed.

19. A storage system for storing data as objects comprising:

a plurality of storage devices; and

a disk controller for controlling said storage devices,

wherein each of said objects includes attribute which defines attributes of the object and data,

wherein said storage devices are configured to implement a Redundant Arrays of Inexpensive Disks (RAID) architecture,

wherein each of said storage devices is formatted to include a plurality of stripes each having a predetermined length and at least one logical unit (LU) is formed by all or a portion of one or a plurality of said storage devices, and

wherein when one of said storage devices fails, a data reconstruction process is implemented such that data recovery is performed on each data stripe where an object or a part of said object is stored and each parity stripe where parity data is stored.

20. The storage system according to claim 19, wherein the RAID architecture being implemented is defined according to a RAID configuration table which includes a plurality of entries each including a port identifier (ID), a logical unit number (LUN), a storage device (disk) ID, a RAID level, and a stripe size, thereby defining for each combination of port ID and LUN, a logical storage device which is formed by all or a portion of one or a plurality of said storage devices and a RAID level and stripe size to be used on said logical storage device.

21. The storage system according to claim 19, wherein said storage system responds to an OSD input/output (I/O) command, and

wherein said OSD I/O command causes said storage system to perform an I/O operation on said object by specifying said logical unit according to a combination of a port identifier (ID) which identifies a particular port through which said logical storage device can be accessed and logical unit number (LUN) which identifies a particular logical unit where said object is stored.

22. The storage system according to claim 19, wherein status of each of said storage devices is monitored by a disk status table which includes a plurality of entries each including a disk ID which identifies a particular storage device, status information and spare disk information, thereby defining for each storage device whether said particular storage device has failed and a hot-standby storage device to be used as a replacement storage device if said particular storage device has failed.

23. The storage system according to claim 22, wherein said status information indicates whether said storage device is operating normally, has failed, or has been replaced by said hot-standby storage device.

24. The storage system according to claim 22, wherein said spare disk information indicates one of the other storage devices to be used as said hot-standby storage device to replace said storage device when said storage device fails.

25. The storage system according to claim 19, wherein a logical-physical (LP) mapping table is used by said storage system to manage relations between objects and stripes, and

wherein said LP mapping table is used to search for a location of an object when said storage system receives an I/O command.

26. The storage system according to claim 25, wherein said LP mapping table includes a plurality of entries each having a port identifier (ID) which identifies a particular port through which a logical storage device can be accessed, a logical unit number (LUN) which identifies a particular logical unit (LU) where an object is stored, a partition ID which identifies a partition on a storage device which forms the LU within which the object is stored, an object ID which identifies the object, and disk location information which identifies in which stripe the object, specified by the partition ID and object ID, is stored.

27. The storage system according to claim 19, wherein a physical-logical (PL) mapping table is used by said storage system to manage relations between disk blocks of each of said storage systems and objects, and

wherein said PL mapping table is used to search which disk block contains a particular object.

28. The storage system according to claim 27, wherein said PL mapping table includes a plurality of entries each having a stripe identifier (ID) which identifies a particular stripe which contains a particular object, a port ID which identifies a particular port through which a logical storage device can be accessed, a logical unit number (LUN) which identifies a particular logical unit (LU) where the object is stored, a partition ID which identifies a partition on a storage device which forms the LU within which the object is stored, and an object ID which identifies the object.

29. The storage system according to claim 21, wherein said OSD I/O command is one of CREATE, WRITE, READ, CREATE AND WRITE, APPEND, GET ATTRIBUTE, and REMOVE commands,

wherein said CREATE command is used to allocate one or more objects in the logical unit,

wherein said WRITE command is used for writing a certain length of data from a specified starting byte address in an object specified by an object ID,

wherein said READ command is used for reading a certain length of data from a specified starting byte address in an object specified with an object ID,

wherein said CREATE AND WRITE command is a compound operation which is used to execute a CREATE operation and a WRITE operation in a single command,

wherein said APPEND command is used to append a specified length of data after a current object,

wherein said GET ATTRIBUTE command is used for retrieving an attribute of an object for use in later processing of the object, and

wherein said REMOVE command is used for deleting an object in a logical storage device.

30. The storage system according to claim 19, wherein when a storage device fails, said storage system creates for said storage device a list of stripes included in said storage device within which objects are stored.

31. The storage system according to claim 30, wherein said list is a disk reconstruction bitmap which includes a plurality of entries each including a stripe identifier (ID) which identifies a particular stripe, recovery information which indicates whether the identified stripe is to be recovered, and done information which indicates whether the data reconstruction of the identified stripe has been completed.

32. The storage system according to claim 30, wherein when a storage device fails said storage system assigns a replacement storage device to replace the failed storage device, starts a process to create said list, and reconstructs data according to said list using a RAID technique.

33. The storage system according to claim 19, wherein during said data reconstruction process isolated stripes each being stored at locations on a storage device separated from other stripes are relocated to be adjacent to the other stripes.

34. The storage system according to claim 33, wherein when a storage device fails, said storage system creates for said storage device a list of stripes including only those stripes within which an object is stored and which are to be reconstructed.

35. The storage system according to claim 34, wherein said list is a disk reconstruction bitmap which includes a plurality of entries each including a stripe identifier (ID) which identifies a particular stripe, new location information which indicates a new location for the reconstructed data of the identified stripe, done information which indicates whether the data reconstruction of the identified stripe has been completed, and parity reconstruction information which indicates when whether a parity stripe corresponding to the identified stripe has been reconstructed.