Method and system for dynamically replicating data within a distributed storage system

• US 8,341,118 B2
• Filed: 02/07/2011
• Issued: 12/25/2012
• Est. Priority Date: 02/09/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Drobychev, Alexandre, Kesselman, Alexander, Vickrey, Rebekah C., Dachille, Frank C., Datuashvili, George
• Primary Examiner(s): LE, DEBBIE M
• Application Number:
  US13/022,579
• Publication Number:
  US 20110196828A1
• Time in Patent Office: 687 Days
• Field of Search: None
• US Class Current: 707/636
• CPC Class Codes: G06F 16/275 Synchronous replication

Abstract

A server computer at a first storage sub-system of a distributed storage system receives from a client a first client request for an object. If the object is not present in the first storage sub-system, the server computer identifies a second storage sub-system of the distributed storage system as having a replica of the requested object, the requested object including content and metadata. The server computer submits an object replication request for the requested object to the second storage sub-system and independently receives the content and metadata of the requested object from the second storage sub-system. The server computer generates a new replica of the object at the first storage sub-system using the received metadata and content and returns the metadata of the new replica of the object to the client.

Claims

1. A computer-implemented method for replicating objects within a distributed storage system, wherein the distributed storage system includes a plurality of storage sub-systems, comprising:

at a first storage sub-system having one or more processors and memory storing one or more programs for execution by the one or more processors,

receiving from a client a first client request for an object that is not present in the first storage sub-system;

identifying a second storage sub-system as having a replica of the requested object, wherein the requested object includes content and metadata;

submitting an object replication request for the requested object to the second storage sub-system, further comprising;

receiving metadata of the requested object from a metadata management component of the second storage sub-system, wherein the metadata includes an extents table of the requested object;

identifying a location of the object content at the second storage sub-system using the extents table;

independently receiving the content and metadata of the requested object from the second storage sub-system;

generating a new replica of the object at the first storage sub-system using the received metadata and content; and

returning the metadata of the new replica of the object to the client.

2. The computer-implemented method of claim 1, further comprising:

upon receipt of the first client request,

extracting an object ID of the requested object from the first client request;

querying a metadata table of the first storage sub-system using the object ID; and

determining whether the object is present in the first storage sub-system in accordance with the query result.

3. The computer-implemented method of claim 1, wherein identifying a second storage sub-system further comprises:

sending a query for the requested object to a third storage sub-system, wherein the third storage sub-system includes metadata of objects stored at the plurality of storage sub-systems; and

receiving a response from the third storage sub-system, wherein the response identifies the second storage sub-system as source and a chunk store within the first storage sub-system as destination.

4. The computer-implemented method of claim 1, wherein submitting an object replication request for the requested object to the second storage sub-system further comprises:

submitting a content replication request to a content management component of the second storage sub-system, wherein the content replication request includes the identified content location of the requested object.

5. The computer-implemented method of claim 4, wherein the content replication request is given a priority higher than other content replication requests that are not triggered by a real-time client request.

6. The computer-implemented method of claim 4, further comprising:

upon receipt of the metadata of the requested object,

generating a metadata entry for the object in a metadata table of the first storage sub-system;

inserting the received metadata into the newly-generated metadata entry of the metadata table;

setting an object state attribute in the newly-generated metadata entry as “

uploading;

” and

returning the newly-generated metadata entry to the requesting client.

7. The computer-implemented method of claim 6, wherein the object content includes one or more chunks, further comprising:

receiving from the client a second client request for accessing a client-specified portion of the object;

for a respective chunk received from the second storage system,

returning the chunk to the requesting client if the chunk overlaps with the client-specified portion of the object;

storing a replica of the chunk within the first storage sub-system; and

updating the metadata entry in the first storage sub-system to reflect the presence of the chunk within the first storage sub-system.

8. The computer-implemented method of claim 7, wherein updating the metadata entry in the first storage sub-system further comprises:

generating a metadata update for the chunk, wherein the metadata update includes location information of the chunk within the first storage sub-system;

updating an extents table of the metadata entry using the location information of the chunk;

updating the object state attribute of the metadata entry to be “

finalized”

if the plurality of chunks of the object are located within one chunk store of the first storage sub-system; and

updating the object state attribute of the metadata entry to be “

finalizing”

if the plurality of chunks of the object are located within multiple chunk stores of the first storage sub-system.

9. The computer-implemented method of claim 8, further comprising:

moving the plurality of chunks from the multiple chunk stores of the first storage sub-system to a destination chunk store of the first storage sub-system; and

updating the object state attribute of the metadata entry to be “

finalized”

if the plurality of chunks of the object are located within the destination chunk store of the first storage sub-system.

10. A computer system for replicating objects within a distributed storage system, wherein the distributed storage system includes a plurality of storage sub-systems, comprising:

one or more processors;

memory for storing one or more programs, wherein the one or more processors are configured to execute at a first storage sub-system the one or more programs including instructions for;

receiving from a client a first client request for an object that is not present in the first storage sub-system;

identifying a second storage sub-system as having a replica of the requested object, wherein the requested object includes content and metadata;

submitting an object replication request for the requested object to the second storage sub-system, further comprising;

receiving metadata of the requested object from a metadata management component of the second storage sub-system, wherein the metadata includes an extents table of the requested object;

identifying a location of the object content at the second storage sub-system using the extents table;

independently receiving the content and metadata of the requested object from the second storage sub-system;

generating a new replica of the object at the first storage sub-system using the received metadata and content; and

returning the metadata of the new replica of the object to the client.

11. The computer system of claim 10, further comprising:

instructions for extracting an object ID of the requested object from the first client request;

instructions for querying a metadata table of the first storage sub-system using the object ID; and

instructions for determining whether the object is present in the first storage sub-system in accordance with the query result.

12. The computer system of claim 10, wherein the instructions for identifying a second storage sub-system further comprise:

instructions for sending a query for the requested object to a third storage sub-system, wherein the third storage sub-system includes metadata of objects stored at the plurality of storage sub-systems; and

instructions for receiving a response from the third storage sub-system, wherein the response identifies the second storage sub-system as source and a chunk store within the first storage sub-system as destination.

13. The computer system of claim 10, wherein the instructions for submitting an object replication request for the requested object to the second storage sub-system further comprise:

instructions for submitting a content replication request to a content management component of the second storage sub-system, wherein the content replication request includes the identified content location of the requested object.

14. The computer system of claim 13, wherein the content replication request is given a priority higher than other content replication requests that are not triggered by a real-time client request.

15. The computer system of claim 13, further comprising:

instructions for generating a metadata entry for the object in a metadata table of the first storage sub-system using the metadata of the requested object received from the second storage sub-system;

instructions for inserting the received metadata into the newly-generated metadata entry of the metadata table;

instructions for setting an object state attribute in the newly-generated metadata entry as “

uploading;

” and

instructions for returning the newly-generated metadata entry to the requesting client.

16. The computer system of claim 15, wherein the object content includes one or more chunks, further comprising:

instructions for receiving from the client a second client request for accessing a client-specified portion of the object;

for a respective chunk received from the second storage system,

instructions for returning the chunk to the requesting client if the chunk overlaps with the client-specified portion of the object;

instructions for storing a replica of the chunk within the first storage sub-system; and

instructions for updating the metadata entry in the first storage sub-system to reflect the presence of the chunk within the first storage sub-system.

17. The computer system of claim 16, wherein the instructions for updating the metadata entry in the first storage sub-system further comprise:

instructions for generating a metadata update for the chunk, wherein the metadata update includes location information of the chunk within the first storage sub-system;

instructions for updating an extents table of the metadata entry using the location information of the chunk;

instructions for updating the object state attribute of the metadata entry to be “

finalized”

if the plurality of chunks of the object are located within one chunk store of the first storage sub-system; and

instructions for updating the object state attribute of the metadata entry to be “

finalizing”

if the plurality of chunks of the object are located within multiple chunk stores of the first storage sub-system.

18. The computer system of claim 17, further comprising:

instructions for moving the plurality of chunks from the multiple chunk stores of the first storage sub-system to a destination chunk store of the first storage sub-system; and

instructions for updating the object state attribute of the metadata entry to be “

finalized”

if the plurality of chunks of the object are located within the destination chunk store of the first storage sub-system.

19. A non-transitory computer readable storage medium storing one or more programs configured for execution by a first storage sub-system having one or more processors and memory storing one or more programs for execution by the one or more processors in association with a distributed storage system that includes a plurality of storage sub-systems, the one or more programs comprising instructions for:

receiving from a client a first client request for an object that is not present in the first storage sub-system;

identifying a second storage sub-system as having a replica of the requested object, wherein the requested object includes content and metadata;

submitting an object replication request for the requested object to the second storage sub-system, further comprising;

receiving metadata of the requested object from the metadata management component of the second storage sub-system, wherein the metadata includes an extents table of the requested object;

identifying a location of the object content at the second storage sub-system using the extents table;

independently receiving the content and metadata of the requested object from the second storage sub-system;

generating a new replica of the object at the first storage sub-system using the received metadata and content; and

returning the metadata of the new replica of the object to the client.

20. The computer readable storage medium of claim 19, further comprising instructions for:

upon receipt of the first client request,

extracting an object ID of the requested object from the first client request;

querying a metadata table of the first storage sub-system using the object ID; and

determining whether the object is present in the first storage sub-system in accordance with the query result.