System and method for recovery from failure of a storage server in a distributed column chunk data store

• US 20070143359A1
• Filed: 12/19/2005
• Published: 06/21/2007
• Est. Priority Date: 12/19/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Uppala, Radha
• Granted Patent: US 7,546,321 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1004 to protect a block of data words, e.g. CRC or checksum G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems

Abstract

An improved system and method for recovery from failure of a storage server in a distributed column chunk data store is provided. A distributed column chunk data store may be provided by multiple storage servers operably coupled to a network. A storage server provided may include a database engine for partitioning a data table into the column chunks for distributing across multiple storage servers, a storage shared memory for storing the column chunks during processing of semantic operations performed on the column chunks, and a storage services manager for striping column chunks of a partitioned data table across multiple storage servers. Any data table may be flexibly partitioned into column chunks using one or more columns with various partitioning methods. Storage servers may then fail and column chunks may be recreated from parity column chunks and redistributed among the remaining storage servers in the column chunk data store.

Claims

1. A computer-implemented method for recovery from failure of a storage server in a distributed system, comprising:

detecting failure of a storage server operably coupled to one or more storage servers storing column chunks of a partitioned data table;

retrieving a parity column chunk stored on a storage server of the one or more storage servers; and

recreating a column chunk from the parity column chunk stored on the storage server of the one or more storage servers.

2. The method of claim 1 further comprising determining whether the parity column chunk for recreating the column chunk may be available on the storage server of the one or more storage servers.

3. The method of claim 1 further comprising receiving a request for retrieving the column chunk from the storage server.

4. The method of claim 1 further comprising returning the recreated column chunk to a sender of the request for retrieving the column chunk.

5. The method of claim 1 further comprising determining whether to send an alert to indicate degraded system performance.

6. The method of claim 1 further comprising sending an alert to indicate degraded system performance.

7. The method of claim 1 further comprising determining whether to redistribute column chunks from the storage server to the one or more storage servers.

8. The method of claim 1 further comprising redistributing the column chunks from the storage server to the one or more storage servers.

9. A computer-readable medium having computer-executable instructions for performing the method of claim 1.

10. A computer-implemented method for recovery from failure of a storage server in a distributed system, comprising:

detecting failure of a storage server operably coupled to one or more storage servers storing column chunks of a partitioned data table;

updating metadata for redistributing column chunks stored on the storage server among the one or more storage servers;

recreating at least one column chunk from a parity column chunk stored on a first storage server of the one or more storage servers; and

storing the at least one recreated column chunk on a second storage server of the one or more storage servers.

11. The method of claim 10 further comprising determining whether to recompute parity of the column chunks of the partitioned data table.

12. The method of claim 10 further comprising computing new parity column chunks for the column chunks.

13. The method of claim 10 further comprising assigning a storage server for storing the at least one column chunk recreated.

14. The method of claim 12 further comprising assigning a different storage server for storing a new parity column chunk than the storage servers assigned for storing the column chunks used to create the new parity column chunk.

15. A computer-readable medium having computer-executable instructions for performing the method of claim 10.

20. The distributed computer system of claim 15 wherein means for redistributing some of the column chunks among the storage server and the one or more storage servers comprises means for moving column chunks stored on the storage server to the one or more storage servers.

16. A distributed computer system for storing data tables, comprising:

means for detecting failure of a storage server operably coupled to one or more storage servers storing column chunks of a partitioned a data table;

means for recreating at least one column chunk from a parity column chunk stored on a storage server of the one or more storage servers; and

means for redistributing some of the column chunks among the storage server and the one or more storage servers.

17. The distributed computer system of claim 16 further comprising means for determining whether to recompute parity of the column chunks.

18. The distributed computer system of claim 16 further comprising means for recomputing parity of the column chunks.

19. The distributed computer system of claim 18 further comprising means for storing a recomputed parity column chunk on a different storage server than the storage servers storing the column chunks used to compute the recomputed parity column chunk.