Replicating data across data storage devices of a logical volume

• US 10,459,891 B2
• Filed: 09/30/2015
• Issued: 10/29/2019
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: Western Digital Technologies Incorporated (Western Digital Corporation)
• Original Assignee: Western Digital Technologies Incorporated (Western Digital Corporation)
• Inventors: Neumann, Charles A.
• Primary Examiner(s): Corrielus, Jean M
• Application Number:
  US14/870,943
• Publication Number:
  US 20170091222A1
• Time in Patent Office: 1,490 Days
• Field of Search: 707636
• CPC Class Codes: G06F 16/1824 implemented using Network-attached Storage [NAS] architecture distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database

Abstract

Systems and methods are disclosed for replicating data across data storage devices of a logical volume. A data storage system may create a logical volume by grouping the multiple data storage devices together. The data storage devices may be external standalone data storage devices. The data storage system may also replicate data across the logical volume.

Claims

1. An apparatus for replicating files across standalone data storage devices, the apparatus comprising:

a communication interface configured to be coupled to a set of external standalone data storage devices; and

a controller coupled to the communication interface, the controller configured to;

create a logical volume comprising the set of external standalone data storage devices;

identify a set of files stored to be replicated across the logical volume;

determine one or more protection levels for the set of files, wherein;

the one or more protection levels are applied across the logical volume; and

the one or more protection levels indicate a desired number of copies to be stored in the logical volume;

replicate the set of files across the logical volume such that a minimum number of copies of the set of files is made on the logical volume based on the one or more protection levels;

in response to determining that a first external standalone data storage device from the set of external standalone data storage devices is to be temporarily removed from the logical volume;

identify a second set of files from the set of files that are replicated on the first external standalone data storage device; and

copy the second set of files on the first external standalone data storage device to other external standalone data storage devices in the set of external standalone data storage devices such that a minimum number of copies for the second set of files is maintained in the logical volume in accordance with one or more protection levels for the second set of files; and

in response to determining that the first external standalone data storage device has been re-inserted into the logical volume;

remove the second set of files from the first external standalone data storage device.

2. The apparatus of claim 1, wherein the controller is further configured to:

monitor the set of files across the logical volume;

determine that one or more copies of a first file from the set of files has been modified; and

update the first file across the logical volume.

3. The apparatus of claim 2, wherein the controller is configured to determine that one or more copies of the first file from the set of files has been modified by:

comparing one or more hashes of the one or more copies of the first file.

4. The apparatus of claim 2, wherein the controller is configured to determine that one or more copies of the first file from the set of files has been modified by comparing one or more of:

timestamps associated with the one or more copies of the first file, or

file sizes associated with the one or more copies of the first file.

5. The apparatus of claim 1, wherein replicating the set of files across the logical volume comprises:

identifying a first file from the set of files, wherein the first file is located on a first external standalone data storage device of the set of external standalone data storage devices;

identifying one or more additional data storage devices from the set of external standalone data storage devices based on a set of storage parameters; and

copying the first file to the one or more additional data storage devices based on the one or more protection levels.

6. The apparatus of claim 5, wherein the controller is further configured to:

update a file list to indicate that the set of files is copied on the one or more additional data storage devices.

7. The apparatus of claim 5, wherein the set of storage parameters comprises one or more of:

a type of the first file;

a size of the first file;

one or more health statuses of the one or more additional data storage devices;

one or more types of the one or more additional data storage devices;

or

a number of times the first file has been accessed.

8. The apparatus of claim 1, wherein creating the logical volume comprises:

aggregating the set of external standalone data storage devices to form the logical volume.

9. The apparatus of claim 1, wherein creating the logical volume comprises:

allocating an amount of space for replicating files on each external standalone data storage device of the set of external standalone data storage devices.

10. The apparatus of claim 1, wherein the set of files is identified based on one or more of:

file types of the set of files;

file sizes of the set of files;

numbers of times the set of files have been accessed;

or

extensions of the set of files.

11. The apparatus of claim 1, wherein the set of files is unique across the logical volume prior to replicating the set of files.

12. The apparatus of claim 1, wherein the controller is further configured to:

receive a request to add an additional external standalone data storage device to the logical volume; and

add the additional external standalone data storage device to the logical volume based on the request.

13. The apparatus of claim 1, wherein the controller is further configured to:

receive a request to remove the first external standalone data storage device from the set of external standalone data storage devices; and

remove the first external standalone data storage device from the set of external standalone data storage devices.

14. The apparatus of claim 1, wherein the controller is further configured to:

determine that the first external standalone data storage device from the set of external standalone data storage devices is inoperable; and

remove the first external standalone data storage device from the set of external standalone data storage devices.

15. The apparatus of claim 1, wherein the controller is further configured to:

cause the set of files to be encrypted.

16. The apparatus of claim 1, wherein the set of external standalone data storage devices comprises:

one or more of a direct-access storage device or a network-access storage device.

17. A method for replicating files across standalone data storage devices, the method comprising:

connecting to a set of external standalone data storage devices;

creating a logical volume comprising the set of external standalone data storage devices;

identifying a set of files stored to be replicated across the logical volume;

determining one or more protection levels for the set of files, wherein;

the one or more protection levels are applied across the logical volume; and

the one or more protection levels indicate a desired number of copies to be stored in the logical volume;

replicating the set of files across the logical volume such that a minimum number of copies of the set of files is made on the logical volume based on the one or more protection levels;

in response to determining that a first external standalone data storage device from the set of external standalone data storage devices is to be temporarily removed from the logical volume;

identifying a second set of files from the set of files that are replicated on the first external standalone data storage device; and

copying the second set of files on the first external standalone data storage device to other external standalone data storage devices in the set of external standalone data storage devices such that a minimum number of copies for the second set of files is maintained in the logical volume in accordance with one or more protection levels for the second set of files; and

in response to determining that the first external standalone data storage device has been re-inserted into the logical volume;

removing the second set of files from the first external standalone data storage device.

18. The method of claim 17, further comprising:

monitoring the set of files across the logical volume;

determining that one or more copies of a first file from the set of files has been modified by comparing one or more hashes of the one or more copies of the first file; and

updating the first file across the logical volume.

19. An apparatus for replicating files across standalone data storage devices, the apparatus comprising:

a communication means configured to be coupled to a set of external standalone data storage devices; and

a processing means coupled to the communication means, the processing means configured to;

create a logical volume comprising the set of external standalone data storage devices;

identify a set of files stored to be replicated across the logical volume;

determine one or more protection levels for the set of files, wherein;

the one or more protection levels are applied across the logical volume; and

the one or more protection levels indicate a desired number of copies to be stored in the logical volume;

replicate the set of files across the logical volume such that a minimum number of copies of the set of files is made on the logical volume based on the one or more protection levels;

in response to determining that a first external standalone data storage device from the set of external standalone data storage devices is to be temporarily removed from the logical volume;

identify a second set of files from the set of files that are replicated on the first external standalone data storage device; and

copy the second set of files on the first external standalone data storage device to other external standalone data storage devices in the set of external standalone data storage devices such that a minimum number of copies for the second set of files is maintained in the logical volume in accordance with one or more protection levels for the second set of files; and

in response to determining that the first external standalone data storage device has been re-inserted into the logical volume;

remove the second set of files from the first external standalone data storage device.

20. The apparatus of claim 19, wherein the processing means is further configured to:

monitor the set of files across the logical volume;

determine that one or more copies of a first file from the set of files has been modified by comparing one or more hashes of the one or more copies of the first file; and

update the first file across the logical volume.