Data deduplication

• US 20120089578A1
• Filed: 08/31/2011
• Published: 04/12/2012
• Est. Priority Date: 08/31/2010
• Status: Active Grant

• Current Assignee: FalconStor, Inc (Hale Capital Management LP)
• Original Assignee: FalconStor, Inc (Hale Capital Management LP)
• Inventors: Lam, Wayne
• Granted Patent: US 9,785,644 B2
• Time in Patent Office: Days
• US Class Current: 707/692
• CPC Class Codes: G06F 16/1752 based on file chunks

Abstract

A method of deduplicating data is disclosed comprising mounting, by a deduplication appliance, network shared storage of a client machine, via a network, accessing data to be deduplicated on the network shared storage device, deduplicating the data, storing the deduplicated data on a second storage device, and replacing the data in the network shared storage device by at least one indicator of the location of the deduplicated data in the second storage device. A method is also disclosed for copying deduplicated data stored by a deduplication appliance, by a client machine, comprising receiving a request to copy data from a first location to a second location, by the client machine, by providing at least one second indicator to the third location on the deduplication appliance, at the second location, if the source and the destination are on the deduplication appliance. Systems are also disclosed.

Claims

1. A method of deduplicating data comprising:

mounting, by a deduplication appliance, network shared storage of a client machine, via a network;

accessing data to be deduplicated on the network shared storage device, by the deduplication appliance;

deduplicating the data, by the deduplication appliance;

storing the deduplicated data on a second storage device by the deduplication appliance; and

replacing the data in the network shared storage device by at least one indicator of the location of the deduplicated data in the second storage device.

2. The method of claim 1, wherein accessing data comprises:

accessing a data file to be deduplicated in the network shared storage device, by the deduplication appliance; and

replacing the data file by at least one stub file containing the at least one indicator.

3. The method of claim 1, comprising:

mounting the first storage device in accordance with at least one rule established by the client machine.

4. The method of claim 3, wherein the rule is chosen from a group consisting of one or more of the following:

identification of the network shared storage of the client machine, identification of directories on the network shared storage to be deduplicated, identification of files on the network shared storage to be deduplicated, the deduplication time, and the deduplication frequency.

5. The method of claim 1, wherein the indicator comprises a pointer.

6. The method of claim 1, further comprising:

receiving a request by the client machine from a user to copy a data file from a first location on the storage device to a second location on the storage device, wherein the data file comprises deduplicated data stored by the deduplication appliance; and

providing at least one second stub file containing at least one indicator to the location of the deduplicated data in the second storage device, if the source and destination of the data to be copied are the deduplication appliance, in response to the request.

7. The method of claim 6, comprising:

providing the at least one second stub file by an agent program, a plug-in module, or a kernel on the client machine.

8. The method of claim 6, wherein the at least one indicator comprises at least one stub file providing at least one second stub file at the second location comprises:

copying the at least one first stub file to the second location.

9. The method of claim 1, wherein the network comprises an intranet, a local area network, a wide area network, a fiber channel storage area network, an Ethernet or the Internet.

10. The method of claim 1, wherein the network shared storage comprises at least a portion of a storage device.

11. A system for deduplicating data, comprising:

a first storage device; and

a deduplication appliance comprising a processor configured to;

mount network shared storage of a client machine, via a network;

access data to be deduplicated on the network shared storage device;

deduplicate the data;

store the deduplicated data on the first storage device; and

replace the data in the network shared storage device by at least one indicator of the location of the deduplicated data in the first storage device.

12. The system of claim 11, wherein the deduplication appliance is configured to:

access a data file to be deduplicated in the network shared storage device;

deduplicate the data in the data file; and

replace the data file by a stub file containing the at least one indicator.

13. The system of claim 12, wherein the deduplication appliance is configured to:

mount the first storage device in accordance with at least one rule established by the client machine.

14. The system of claim 13, wherein the rule is chosen from a group consisting of one or more of the following:

identification of the network shared storage of the client machine, identification of directories on the network shared storage to be deduplicated, identification of files on the network shared storage to be deduplicated, the deduplication time, and the deduplication frequency.

15. The system of claim 11, wherein the at least one indicator comprises at least one pointer.

16. The method of claim 11, wherein the deduplication appliance is further configured to:

provide a program to a client machine, the program configured to;

in response to a request from a client to copy data from a first location to a second location on the client machine, determine whether a source and a destination of the data to be copied is on the deduplication appliance; and

provide at least one second indicator to the location of the deduplicated data in the second storage device at the second location, if the source and destination of the data are on the deduplication appliance.

17. The system of claim 16, wherein the at least one second indicator is contained in at least one stub file, and program is configured to:

create at least one second stub file in the second location containing the at least one second indicator.

18. The system of claim 16, wherein the program comprises an agent program, a plug-in module, or a kernel on the client machine.

19. The system of claim 11, wherein the network comprises an intranet, a local area network, a wide area network, a fiber channel storage area network, an Ethernet or the Internet;

20. The system of claim 11, wherein the network shared storage comprises at least a portion of a storage device.

24. The method of claim 11, wherein, prior to receiving the request, the data in the first location is deduplicated and replaced by the at least on indicator to the third location on the deduplication appliance.

25. The method of claim 24, comprising:

providing data to be deduplicated to the deduplication appliance by the client machine, via a network.

26. The method of claim 25, wherein the first location and the second location are in network shared storage of the deduplication appliance, the network shared storage being accessible to the client machine via a network.

27. The method of claim 25, wherein at least a portion of the first storage device containing the data is a network shared storage, wherein the data is accessed by the deduplication appliance via the network shared storage.

21. A method of copying deduplicated data stored by a deduplication appliance, by a client machine, the method comprising:

receiving a request to copy data from a first location to a second location, by the client machine;

determine whether the data in the first location has a source on a deduplication appliance by determining whether the first location includes an indicator to a third location on the deduplication appliance where the data is stored;

determining whether the second location has a destination on the deduplication appliance; and

providing at least one second indicator to the third location on the deduplication appliance, at the second location, if the source and the destination are on the deduplication appliance.

22. The method of claim 21, wherein the request to copy data comprises a request to copy a data file, and the at least one first indicator is in at least one stub file, the method comprising:

providing at least one second stub file containing at least one second indicator to the second location, if the source and the destination are on the deduplication appliance.

23. The method of claim 22, wherein providing the at least one second stub file comprises:

copying the at least one first stub file to the second location.

28. The method of claim 21, comprising:

determining and providing by an agent program, a plug-in module, or a kernel on the client machine.

29. A system for copying data deduplicated by a deduplication appliance, the system comprising:

at least one first storage device; and

a processor configured to;

receive a request to copy data from a first location to a second location on the at least one first storage device;

determine whether the data at the first location has a source on a deduplication appliance by determining whether the first location includes an indicator to a third location on the deduplication appliance where the data is stored;

determine whether second location has a destination on the deduplication appliance; and

provide at least one second indicator to the third location if the source and the destination are on the deduplication appliance.

30. The system of claim 29, wherein the request for the data comprises a request for a data file, and the processor is configured to:

copy the at least one first stub file to the second location, if the source and the destination are on the deduplication appliance.

31. The system of claim 29, wherein the processor is configured to:

provide data to be deduplicated to the deduplication appliance, via a network.

32. The system of claim 31, wherein the first location and the second location are in network shared storage of the deduplication appliance, accessible to the client machine via a network.

33. The system of claim 29, wherein at least a portion of the first storage device comprises network shared storage accessible by the deduplication appliance over a network, and the first and second locations are in the network shared storage.

34. The system of claim 29, wherein the processor is configured to:

provide the at least one second indicator by an agent program, a plug-in module, or a kernel.

35. A deduplication appliance comprising:

storage to store deduplicated data; and

a processor configured to;

provide to a client machine software configured to;

monitor for a request to copy data from a first location to a second location on the client machine;

determine whether the first location has a source on the deduplication appliance by determining whether the first location includes an indicator to a third location on the deduplication appliance where the data is stored;

determine whether second location has a destination on the deduplication appliance; and

provide at least one second indicator to the third location if the source and the destination are on the deduplication appliance.