SUB-CLUSTER RECOVERY USING A PARTITION GROUP INDEX

• US 20170249087A1
• Filed: 02/27/2017
• Published: 08/31/2017
• Est. Priority Date: 02/25/2016
• Status: Active Grant

• Current Assignee: Rubrik, Inc.
• Original Assignee: Datos Io Inc. (Rubrik, Inc.)
• Inventors: Shekhar, Rohit, Kim, Hyo Jun, Sarkar, Prasenjit, Lu, Maohua, Raghavan, Ajaykrishna, Zhou, Pin
• Granted Patent: US 10,852,998 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Embodiments disclosed herein provide systems, methods, and computer readable media for sub-cluster recovery in a data storage environment having a plurality of storage nodes. In a particular embodiment, the method provides scanning data items in the plurality of nodes. While scanning, the method further provides indexing the data items into an index of a plurality of partition groups. Each partition group includes data items owned by a particular one of the plurality of storage nodes. The method then provides storing the index.

Claims

1. A method for sub-cluster recovery in a data storage environment having a plurality of nodes, the method comprising:

scanning data items in the plurality of nodes;

while scanning, indexing the data items into an index of a plurality of partition groups, wherein each partition group comprises data items owned by a particular one of the plurality of nodes; and

storing the index.

2. The method of claim 1, further comprising:

using the index to recover the plurality of nodes by directly generating per node data from partition group to node mapping in the index.

3. The method of claim 1, further comprising:

using the index to recover a second plurality of nodes less in number than the plurality of nodes by generating per node data for partial cluster from partition group to node mapping in the index.

4. The method of claim 1, further comprising:

using the index to recover a second plurality of nodes different than the plurality of nodes by generating a partition group to node mapping for the new cluster from the index and generating data per node of the new cluster using the mapping.

5. The method of claim 1, further comprising:

while scanning the data items, identifying duplicate data items;

deduplicating and repackaging each of the duplicate data items into respective deduplicated data units; and

storing the deduplicated data units to a secondary data repository.

6. The method of claim 5, further comprising:

checking the duplicate data items for consistency across the plurality of nodes.

7. The method of claim 6, wherein only consistent data items are included in the duplicate data items.

8. The method of claim 5, further comprising:

before storing the deduplicated data units to the secondary data repository, compiling the deduplicated data units into a data version of the data items; and

wherein storing the deduplicated data units comprises storing the data version.

9. The method of claim 1, wherein the plurality of nodes comprises a NoSQL data store.

10. A system for sub-cluster recovery in a data storage environment having a plurality of nodes, the system comprising:

one or more computer readable storage media;

a processing system operatively coupled with the one or more computer readable storage media; and

program instructions stored on the one or more computer readable storage media that, when read and executed by the processing system, direct the processing system to;

scan data items in the plurality of nodes;

while scanning, index the data items into an index of a plurality of partition groups, wherein each partition group comprises data items owned by a particular one of the plurality of nodes; and

store the index.

11. The system of claim 10, wherein the program instructions further direct the processing system to:

use the index to recover the plurality of nodes by directly generating per node data from partition group to node mapping in the index.

12. The system of claim 10, wherein the program instructions further direct the processing system to:

use the index to recover a second plurality of nodes less in number than the plurality of nodes by generating per node data for partial cluster from partition group to node mapping in the index.

13. The system of claim 10, wherein the program instructions further direct the processing system to:

use the index to recover a second plurality of nodes different than the plurality of nodes by generating a partition group to node mapping for the new cluster from the index and generating data per node of the new cluster using the mapping.

14. The system of claim 10, wherein the program instructions further direct the processing system to:

while scanning, identify duplicate data items;

deduplicate and repackage each of the duplicate data items into respective deduplicated data units; and

store the deduplicated data units to a secondary data repository.

15. The system of claim 14, wherein the program instructions further direct the processing system to:

check the duplicate data items for consistency across the plurality of nodes.

16. The system of claim 15, wherein only consistent data items are included in the duplicate data items.

17. The system of claim 14, wherein the program instructions further direct the processing system to:

before storing the deduplicated data units to the secondary data repository, compile the deduplicated data units into a data version of the data items; and

wherein the program instructions direct the processing system to store the deduplicated data units by directing the processing system to store the data version.

18. The system of claim 10, wherein the plurality of nodes comprises a NoSQL data store.