METHOD, APPARATUS AND SYSTEM FOR DATA DEDUPLICATION

• US 20130311434A1
• Filed: 11/17/2011
• Published: 11/21/2013
• Est. Priority Date: 11/17/2011
• Status: Abandoned Application

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Jones, Marc T.
• Application Number:
  US13/997,966
• Publication Number:
  US 20130311434A1
• Time in Patent Office: Days
• US Class Current: 707/692
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Techniques and mechanisms for limiting storage of duplicate data in a storage back-end. In an embodiment, a storage device of the storage back-end receives from a storage front-end a write command specifying a write of data to the storage back-end. In another embodiment, the storage device calculates and provides to the storage front-end a data signature for data which is the subject of the write command. Based on the data signature provided by the storage device, a deduplication engine of the storage front-end determines whether a deduplication operation is to be performed.

Claims

1. A method at a first computer platform providing a storage front-end, the method comprising:

sending a write command from the storage front-end to a storage device of a storage back-end, the write command specifying a write of first data to the storage device;

receiving from the storage device a data fingerprint for the first data, the data fingerprint calculated by the storage device in response to the write command;

in response to receiving the data fingerprint, determining whether a deduplication operation is to be performed; and

if the first data is determined to be a duplicate of other data stored in the storage back-end, signaling that the deduplication operation is to be performed.

2. The method of claim 1, wherein the storage font-end includes at least one of:

a process executing on a processor of the first computer platform; and

one or more components of a chipset of the first computer platform;

wherein the storage back-end is coupled to the processor and the chipset via a hardware interface.

3. The method of claim 2, wherein a second computer platform coupled to the first computer platform includes the storage device.

4. The method of claim 1, wherein determining whether the deduplication operation is to be performed includes:

accessing a repository including one or more data fingerprints each representing respective data stored in the storage back-end, and

searching the repository to determine whether any of the one or more data fingerprints of the repository matches the data fingerprint for the first data.

5. The method of claim 1, wherein the storage device is a component of the first computer platform, the method further comprising:

receiving the write command at the storage device;

calculating the data fingerprint with the storage device in response to receiving the write command; and

with the storage device, sending the data fingerprint to the storage front-end.

6. The method of claim 5, wherein the write command is exchanged according to a communication protocol, wherein sending the data fingerprint includes the storage device sending to the storage front-end a response message corresponding to the write command, the response message according to the communication protocol.

7. The method of claim 1, wherein the deduplication operation includes one of:

deleting the first data from a first memory location; and

deleting metadata indicating that the first data is stored in the first memory location.

8. A computer system for providing a storage front-end, the computer system comprising:

a protocol engine of the storage front-end, the protocol engine to send a write command to a storage device of a storage back-end, the write command to specify a write of first data to the storage device;

a deduplication engine of the storage front-end, the deduplication engine to receive from the storage device a data fingerprint for the first data, the data fingerprint calculated by the storage device in response to the write command, the deduplication engine further to determine, based on the received data fingerprint, whether a deduplication operation is to be performed, wherein, if the first data is determined to be a duplicate of other data stored in the storage back-end, the deduplication engine further to signal that the deduplication operation is to be performed.

9. The computer system of claim 8, wherein the storage front-end includes at least one of:

a process executing on a processor of a computer system; and

one or more components of a chipset of the computer system;

wherein the storage back-end is coupled to the processor and the chipset via a hardware interface.

10. The computer system of claim 9, wherein the computer system is coupled to a computer platform including the storage device.

11. The computer system of claim 8, wherein the deduplication engine to determine whether the deduplication operation is to be performed includes:

the deduplication engine to access a repository including one or more data fingerprints each representing respective data stored in the storage back-end; and

the deduplication engine to search the repository to determine whether any of the one or more data fingerprints of the repository matches the data fingerprint for the first data.

12. The computer system of claim 8, further comprising the storage device, wherein the storage device includes:

protocol logic to receive the write command; and

fingerprint generator logic coupled to the protocol logic, the fingerprint generator logic to calculate, in response to the write command, the data fingerprint for the first data;

wherein the protocol logic further to send the data fingerprint to the storage front-end.

13. The computer system of claim 8, wherein the deduplication operation includes one of:

deleting the first data from the first memory location; and

deleting metadata indicating that the first data is stored in the first memory location.

14. The computer system of claim 8, wherein the write command is exchanged according to a communication protocol, wherein communicating the data fingerprint includes the storage device sending to the storage front-end a response message corresponding to the write command, the response message according to the communication protocol.

18. The storage device of claim 13, wherein the storage device is to operate as a component of a second computer platform coupled to the first computer platform.

15. A storage device including:

protocol logic to receive a write command sent from a storage front-end, the write command specifying a write of first data to the storage device; and

fingerprint generator logic coupled to the protocol logic, the fingerprint generator logic to calculate, in response to the received write command, a data fingerprint for the first data

wherein the protocol logic further to communicate the data fingerprint to the storage front-end; and

wherein, in response to communication of the data fingerprint, a deduplication engine of the storage front-end determines whether a deduplication operation is to be performed.

16. The storage device of claim 15, wherein the storage front-end includes at least one of:

a process executing on a processor of a first computer platform; and

one or more components of a chipset of the first computer platform;

wherein the storage back-end is to couple to the processor and the chipset via a hardware interface.

17. The storage device of claim 16, wherein the storage device is to operate as a component of the first computer platform.

19. The storage device of claim 15, wherein the deduplication engine determines, after the first data is stored in a first memory location in the storage device, that the deduplication operation is to be performed, and wherein the deduplication operation includes one of:

deleting the first data from the first memory location; and

deleting metadata indicating that the first data is stored in the first memory location.

20. The storage device of claim 15, wherein the write command is exchanged according to a communication protocol, wherein communicating the data fingerprint includes the storage device sending to the storage front-end a response message corresponding to the write command, the response message according to the communication protocol.