System and method for re-striping a set of objects onto an exploded array of storage units in a computer system

• US 5,933,834 A
• Filed: 10/16/1997
• Issued: 08/03/1999
• Est. Priority Date: 10/16/1997
• Status: Expired due to Term

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: International Business Machines Incorporated (International Business Machines Corporation)
• Inventors: Aichelen, Tyky
• Primary Examiner(s): Lintz, Paul R.
• Assistant Examiner(s): Colbert, Ella
• Application Number:
  US08/951,967
• Time in Patent Office: 656 Days
• Field of Search: 707/103, 707/2, 707/4, 711/4, 711/100, 711/101, 711/112, 711/114, 711/206, 711/209, 711/2, 711/170, 395/181, 395/182.04, 395/200.49, 364/236.2, 364/268.3, 364/268, 348/7
• US Class Current: N/A
• CPC Class Codes: G06F 11/1096 Parity calculation or recalculation after configuration or reconfiguration of the system
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0655 Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04N 21/21 Server components or server architectures
• CPC Class Codes: H04N 21/23 Processing of content or additional data; Elementary server operations; Server middleware
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure

Abstract

In a computer system having a processor, memory, and a set of storage units storing a set of objects, each object comprising a set of segments arranged in a set of original object stripes on separate storage units, a directory is maintained identifying an object name and a location of each stripe of the object in the set of storage units. A method is provided for re-striping the set of objects onto an expanded set of storage units comprising the set of storage units and at least one other additional storage unit. A list is maintained of each object to be copied onto the expanded set of storage units. Each segment is copied to a new set of stripes on a separate storage unit of the expanded set to create a new object while continuing to store an original copy of each segment of the list object in the original set of object stripes. The directory is updated to include a new object name and a new location of each new stripe of the new object. After all the list object segments are copied to the new stripes, the original copy of each list object segment is deleted and the new object is renamed in the directory to the list object name.

Claims

1. In a computer system having a processor, memory and a set of storage units storing a set of objects, each object comprising a set of segments arranged in an original set of object stripes on separate storage units, wherein a directory is maintained identifying an object name and a location of each object stripe in the set of storage units, a method for re-striping the set of objects onto an expanded set of storage units comprising the set of storage units and at least one additional storage unit, comprising the steps of:

a) maintaining a list of each object to be copied onto the expanded set of storage units;

b) copying each segment of a list object in block units to a new set of new stripes on separate storage units of the expanded set to create a new object while continuing to store an original copy of each segment of the list object in the original set of object stripes;

c) updating the directory to include a new object name, and a new location of each new stripe of the new object;

d) after all of the list object segments are copied to the new stripes, deleting the original copy of each list object segment from the original set of object stripes and renaming, in the directory, the new object to the list object name; and

e) deleting a list object from the list of each object to be copied once all of the list object segments have been copied to the new stripes and the object is renamed to the list object name.

2. The method of claim 1 further comprising the steps of:

identifying whether there are sufficient available storage locations in each storage unit of the expanded set in which to copy the segments of the list object; and

when there are not sufficient available storage locations, placing the list object on a delay list of objects to be copied at a later time, and designating a next object on the list of each object to be copied as the list object.

3. The method of claim 1 wherein each object is a video, each stripe is a clip of the video of a set duration and each storage unit is an array of disk drives.

4. The method of claim 1 wherein the segments are striped sequentially across the storage units.

5. The method of claim 1 wherein an access request for a requested object is executed during the re-striping of the segments of the requested object.

6. The method of claim 1 wherein when a list object is deleted while it is being copied, the new object is also deleted.

7. A computer storage and retrieval system comprising:

a set of storage units storing sets of data objects, each data object comprising segments stored in stripes on separate storage units wherein an additional storage unit is added to the set of storage units to produce an expanded set of storage units;

a directory stored in a memory cataloging each object name and a location for each object stripe;

a listing maintained in memory of each object to be copied onto the expanded set of storage units;

means for identifying whether there is sufficient available storage locations in the expanded set in which to copy a list object and designating a next object on the list of each object to be copied as the list object, when there is not sufficient available storage location to copy the list object;

means for copying each segment of the list object to a new set of new stripes on separate storage units of the expanded set to create a new object while continuing to store an original copy of each segment of the list object in the set of object stripes;

means for updating the directory to include a new object name, and a new location of each new stripe of the new object;

means for deleting the original copy of each list object segment after all of the list object segments are copied to the new stripes;

means for renaming, in the directory, the new object to the list object names;

means for deleting the new object when the list object is deleted while it is being copied; and

means for satisfying an access request for the list object during the re-striping of the segments of the list object.

8. The system of claim 7 further comprising:

means for placing the list object on a delay list of objects to be copied at a later time when there is not sufficient available storage location to copy the list object; and

means for deleting a list object from the list of each object to be copied once all of the list object segments have been copied to the new stripes.

9. The method of claim 7 wherein the computer storage and retrieval system is a video server, the objects are a video and each stripe is a clip of a predetermined duration.

10. The method of claim 7 wherein the segments are stored sequentially across the set of storage devices such that there are x stripes, segments numbered 1, x+1, 2*x+1, and so forth are the first stripe, segments numbered 2, x+2, 2*x+2 are the second stripe, and so forth.

11. An article of manufacture for use in a computer system for re-striping a set of objects stored on an original set of stripes on separate storage units onto a new set of stripes on an expanded set of storage units, the expanded set including the set of storage units and at least one additional storage unit, said article of manufacture comprising a computer-readable storage medium having computer program code embodied in said medium which causes the computer system to:

maintain a list of each object to be copied onto the expanded set of storage units;

copy each segment of a list object to a new set of new stripes on separate storage units of the expanded set to create a new object while continuing to store an original copy of each segment of the list object in the set of object stripes;

update an object directory to include a new object name, and a new location of each new stripe of the new object;

after all of the list object segments are copied to the new stripes, delete the original copy of each list object segment from the original set of stripes and renaming, in the object directory, the new object to the list object name;

identify whether there is sufficient available storage locations in the expanded set in which to copy the list object;

when there is not sufficient available storage location, place the list object on a delay list of objects to be copied at a later time, and designate a next object on the list of each object to be copied as the list object;

delete the list object from the list of each object to be copied once all of the list object segments have been copied to the new stripes; and

delete the new object when a list object is deleted while it is being copied.

12. The article of manufacture of claim 11 wherein the computer system is a video server, the objects are videos and each stripe is a clip of the video of a predetermined duration.

13. The article of manufacture of claim 11 wherein the computer program code further causes the computer system to stripe the segments sequentially across the storage units.

14. The article of manufacture of claim 11 wherein the computer program code further causes the computer system to execute an access request for a requested object during the re-striping of the segments of the requested object.