Distributed file system in a distributed deduplication data storage system

• US 10,592,357 B2
• Filed: 10/20/2016
• Issued: 03/17/2020
• Est. Priority Date: 12/30/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Vijayan, Manoj Kumar, Kochunni, Jaidev Oppath, Attarde, Deepak Raghunath, Ankireddypalle, Ramachandra Reddy
• Primary Examiner(s): Wilson, Yolanda L
• Application Number:
  US15/299,298
• Publication Number:
  US 20170192861A1
• Time in Patent Office: 1,244 Days
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0632 by initialisation or re-initialisation of storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 : Plurality of storage devices
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Systems and methods are provided herein for automatically configuring newly installed secondary storage computing devices and managing secondary storage computing devices when one or more become unavailable. For example, a storage manager can then detect the computing resources available to the newly installed secondary storage computing device, assign a role to the newly installed secondary storage computing device based on the detected computing resources, configure the newly installed secondary storage computing device with deduplication and storage policies used by the other secondary storage computing devices, re-partition secondary storage devices to allocate memory for the newly installed secondary storage computing device, and instruct other secondary storage computing devices to replicate their managed data such that the newly installed secondary storage computing device has access to the replicated data.

Claims

1. A networked information management system configured to automatically configure installed data storage computers, the networked information management system comprising:

a first data storage computer comprising first computer hardware, the first data storage computer configured to generate secondary copies of first primary data;

a first secondary storage device managed by the first data storage computer, wherein the first secondary storage device comprises a first hard disk and a second hard disk;

a second data storage computer comprising second computer hardware, the second data storage computer configured to generate secondary copies of second primary data;

a second secondary storage device managed by the second data storage computer, wherein the second secondary storage device comprises a third hard disk and a fourth hard disk;

a third data storage computer comprising third computer hardware, the third data storage computer configured to generate secondary copies of third primary data, wherein the third data storage computer is installed in the networked information management system after the first data storage computer and the second data storage computer;

a third secondary storage device managed by the third data storage computer, wherein the third secondary storage devices comprises a fifth hard disk and a sixth hard disk; and

a storage manager comprising computer hardware configured to;

detect that the third data storage computer is installed in the networked information management system,

associate the first hard disk, the third hard disk, and the fifth hard disk such that the first hard disk, the third hard disk, and the fifth hard disk together store secondary copy data corresponding to a first file system, and

associate the second hard disk, the fourth hard disk, and the sixth hard disk such that the second hard disk, the fourth hard disk, and the sixth hard disk together store secondary copy data corresponding to a second file system,

wherein the first data storage computer continues to receive input/output (I/O) requests for the second hard disk after a failure of the first hard disk.

2. The networked information management system of claim 1, wherein the storage manager is further configured to redirect a first I/O request intended for the first data storage computer and the first file system to one of the second data storage computer or the third data storage computer after the failure of the first hard disk.

3. The networked information management system of claim 2, wherein the second data storage computer is configured to process the first I/O request using the third hard disk.

4. The networked information management system of claim 2, wherein the third data storage computer is configured to process the first I/O request using the fifth hard disk.

5. The networked information management system of claim 1, wherein the storage manager is further configured to direct a first I/O request intended for the second file system to the first data storage computer after the failure of the first hard disk, after a failure of the fourth hard disk, or after a failure of the sixth hard disk.

6. The networked information management system of claim 1, wherein the failure of the first hard disk does not result in a failure of the first data storage computer.

7. The networked information management system of claim 1, wherein the storage manager is further configured to perform a parity replication operation on the first, third, and fifth hard disks.

8. A computer-implemented method for automatically configuring installed data storage computers, the computer-implemented method comprising:

detecting a presence of a first data storage computer and a second data storage computer, wherein the first data storage computer is configured to generate secondary copies of first primary data, wherein the second data storage computer is configured to generate secondary copies of second primary data, wherein a first secondary storage device is managed by the first data storage computer, wherein the first secondary storage device comprises a first hard disk and a second hard disk, wherein a second secondary storage device is managed by the second data storage computer, and wherein the second secondary storage device comprises a third hard disk and a fourth hard disk;

detecting that a third data storage computer is installed, wherein the third data storage computer is configured to generate secondary copies of third primary data, wherein the third data storage computer is installed after the first data storage computer and the second data storage computer, wherein a third secondary storage device is managed by the third data storage computer, and wherein the third secondary storage devices comprises a fifth hard disk and a sixth hard disk;

associating the first hard disk, the third hard disk, and the fifth hard disk such that the first hard disk, the third hard disk, and the fifth hard disk together store secondary copy data corresponding to a first file system; and

associating the second hard disk, the fourth hard disk, and the sixth hard disk such that the second hard disk, the fourth hard disk, and the sixth hard disk together store secondary copy data corresponding to a second file system,

wherein the first data storage computer continues to receive input/output (I/O) requests for the second hard disk after a failure of the first hard disk.

9. The computer-implemented method of claim 8, further comprising redirecting a first I/O request intended for the first data storage computer and the first file system to one of the second data storage computer or the third data storage computer after a failure of the first hard disk.

10. The computer-implemented method of claim 9, wherein the second data storage computer is configured to process the first I/O request using the third hard disk.

11. The computer-implemented method of claim 9, wherein the third data storage computer is configured to process the first I/O request using the fifth hard disk.

12. The computer-implemented method of claim 8, further comprising directing a first input/output (I/O) request intended for the second file system to the first data storage computer after at least one of a failure of the first hard disk, a failure of the fourth hard disk, or a failure of the sixth hard disk.

13. The computer-implemented method of claim 8, wherein a failure of the first hard disk does not result in a failure of the first data storage computer.

14. The computer-implemented method of claim 8, further comprising performing a parity replication operation on the first, third, and fifth hard disks.

15. A networked information management system configured to automatically configure installed data storage computers, the networked information management system comprising:

a first secondary storage device managed by a first data storage computer configured to generate first secondary copies, wherein the first secondary storage device comprises a first hard disk and a second hard disk;

a second secondary storage device managed by a second data storage computer configured to generate second secondary copies, wherein the second secondary storage device comprises a third hard disk and a fourth hard disk;

a third secondary storage device managed by a third data storage computer configured to generate third secondary copies, wherein the third secondary storage devices comprises a fifth hard disk and a sixth hard disk; and

a storage manager comprising computer hardware configured to;

detect that the third data storage computer is installed in the networked information management system,

associate the first hard disk, the third hard disk, and the fifth hard disk such that the first hard disk, the third hard disk, and the fifth hard disk together store secondary copy data corresponding to a first file system, and

associate the second hard disk, the fourth hard disk, and the sixth hard disk such that the second hard disk, the fourth hard disk, and the sixth hard disk together store secondary copy data corresponding to a second file system,

wherein the first data storage computer continues to receive input/output (I/O) requests for the second hard disk after a failure of the first hard disk.

16. The networked information management system of claim 15, wherein the storage manager is further configured to redirect a first I/O request intended for the first data storage computer and the first file system to one of the second data storage computer or the third data storage computer after a failure of the first hard disk.

17. The networked information management system of claim 16, wherein the second data storage computer is configured to process the first I/O request using the third hard disk.

18. The networked information management system of claim 16, wherein the third data storage computer is configured to process the first I/O request using the fifth hard disk.

19. The networked information management system of claim 15, wherein the storage manager is further configured to direct a first I/O request intended for the second file system to the first data storage computer after at least one of a failure of the first hard disk, a failure of the fourth hard disk, or a failure of the sixth hard disk.

20. The networked information management system of claim 15, wherein the storage manager is further configured to perform a parity replication operation on the first, third, and fifth hard disks.