METHODS AND SYSTEMS FOR IMPROVED THROUGHPUT PERFORMANCE IN A DISTRIBUTED DATA DE-DUPLICATION ENVIRONMENT

• US 20090254609A1
• Filed: 04/08/2008
• Published: 10/08/2009
• Est. Priority Date: 04/08/2008
• Status: Active Grant

• Current Assignee: Quantum Corporation (Chi Ko Investment Co., Ltd.)
• Original Assignee: Quantum Corporation (Chi Ko Investment Co., Ltd.)
• Inventors: Wideman, Roderick B.
• Granted Patent: US 8,751,561 B2
• Time in Patent Office: Days
• US Class Current: 709/203
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

In accordance with some embodiments, of the systems and methods described here a data storage system that may include data de-duplication may receive a stream of data and parse the stream of data into a block at a local client node. Additionally, in some embodiments, a code that represents the block of data might be determined at the local client node. This code, representing the block of data, may be sent to a server. In accordance with various embodiments, the server may determine if a block is unique, for example, based on the code received at the server. In various embodiments, the server might write a unique block to a file at the local client node; and update metadata.

Claims

1. A method of data storage in a data de-duplication system comprising:

parsing a stream of data into a block at a local client node;

determining, at the local client node, a code that represents the block of data;

sending the code representing the block of data to a server;

receiving, from a server, a determination that the block is unique based on the code; and

writing the block to a file at the local client node based on the determination that the block is unique.

2. The method of claim 1, further comprising updating metadata at the server.

3. The method of claim 1, wherein the code is sent from the local client node to the server over a local area network.

4. The method of claim 1, further comprising the server notifying the client node if a block is unique.

5. The method of claim 4, wherein the notification is sent from the server to the local client over a local area network.

6. The method of claim 1, wherein writing a unique block to a file at the local client comprises writing to a cluster file.

7. The method of claim 6, wherein the writing a unique block to a file at the local client comprises writing directly to a local file system client.

8. The method of claim 7, wherein the local file system client writes to a disk drive.

9. The method of claim 6, wherein the writing a unique block at the local client comprises writing directly to a storage device.

10. The method of claim 1, wherein the server resolves any file access and the local client node provides that data block to a system requesting access.

11. A data storage system comprising:

a server configured to determine if a block is unique based on a code received at the server from a local client node; and

the local client node configured to;

parse the stream of data into a block at the local client node;

determine the code that represents the block of data;

send the code representing the block of data to the server; and

write the block to a file at the local client node based on the determination that the block is unique.

12. The data storage system of claim 11, wherein the code comprises a hash.

13. The data storage system of claim 11, wherein the code is sent from the local client node to the server over a local area network.

14. The data storage system of claim 11, wherein the server is further configured to notify the client node if a block is unique.

15. The data storage system of claim 14, wherein the notification is sent from the server to the local client over a local area network.

16. The data storage system of claim 11, wherein writing a unique block to a file at the local client comprises writing to a cluster file.

17. The data storage system of claim 16, wherein the writing a unique block to a file at the local client comprises writing directly to a local file system client.

18. The data storage system of claim 17, wherein the local file system client writes to a disk drive.

19. The data storage system of claim 16, wherein the writing a unique block to a file at the local client comprises writing directly to a storage device.

20. The data storage system of claim 11, wherein the server resolves any file access and the local client node provides that data block to a system requesting access.