SYSTEM AND METHOD FOR ACCELERATING ANCHOR POINT DETECTION
First Claim
1. A method for anchor detection, the method comprising the steps of:
- detecting, by anchor detection circuitry, a location of a set of anchors in a received data set;
transferring the location of the set of anchors to a de-duplication module;
identifying, by the de-duplication module, duplicate data within the received data set; and
removing the identified duplicate data from the received data set prior to storage of the received data set on a storage device.
2 Assignments
0 Petitions
Accused Products
Abstract
A sampling based technique for eliminating duplicate data (de-duplication) stored on storage resources, is provided. According to the invention, when a new data set, e.g., a backup data stream, is received by a server, e.g., a storage system or virtual tape library (VTL) system implementing the invention, one or more anchors are identified within the new data set. The anchors are identified using a novel anchor detection circuitry in accordance with an illustrative embodiment of the present invention. Upon receipt of the new data set by, for example, a network adapter of a VTL system, the data set is transferred using direct memory access (DMA) operations to a memory associated with an anchor detection hardware card that is operatively interconnected with the storage system. The anchor detection hardware card may be implemented as, for example, a FPGA is to quickly identify anchors within the data set. As the anchor detection process is performed using a hardware assist, the load on a main processor of the system is reduced, thereby enabling line speed de-duplication.
-
Citations
24 Claims
-
1. A method for anchor detection, the method comprising the steps of:
-
detecting, by anchor detection circuitry, a location of a set of anchors in a received data set; transferring the location of the set of anchors to a de-duplication module; identifying, by the de-duplication module, duplicate data within the received data set; and removing the identified duplicate data from the received data set prior to storage of the received data set on a storage device. - View Dependent Claims (2, 3)
-
-
4. A method for anchor detection, the method comprising the steps of:
-
receiving a new data set; transferring the new data set to anchor detection circuitry; detecting, by the anchor detection circuitry, a location of a set of anchors in the new data set; and transferring the location of the set of anchors to a de-duplication module. - View Dependent Claims (5, 6, 7, 8, 9)
-
-
10. A system for anchor detection, the system comprising:
anchor detection hardware configured to (i) receive a data stream, (ii) identify a set of anchor locations in the data stream, and (iii) forward the set of anchor locations to a de-duplication module. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17, 18, 19)
-
20. A system for anchor detection, the system comprising:
-
means for receiving a new data set; means for transferring the new data set to anchor detection hardware; means for detecting, by the anchor detection hardware, a location of a set of anchors in the new data set; and means for transferring the location of the set of anchors to a de-duplication module. - View Dependent Claims (21)
-
-
22. A method for anchor detection, the method comprising the steps of:
-
transferring a data set to anchor detection hardware; detecting, by the anchor detection hardware, a location of a set of anchors in the new data set; transferring the location of the set of anchors to a de-duplication module; and de-duplicating the data set using the set of anchors in the data set. - View Dependent Claims (23, 24)
-
Specification