BACKGROUND DEDUPLICATION OF DATA SETS IN A MEMORY

• US 20130326115A1
• Filed: 05/31/2012
• Published: 12/05/2013
• Est. Priority Date: 05/31/2012
• Status: Active Grant

• Current Assignee: Seagate Technology LLC (Seagate Technology Holdings Plc)
• Original Assignee: Seagate Technology LLC (Seagate Technology Holdings Plc)
• Inventors: Goss, Ryan James, Gaertner, Mark Allen, Seekins, David Scott
• Granted Patent: US 8,930,612 B2
• Time in Patent Office: Days
• US Class Current: 711/103
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 2212/7205 Cleaning, compaction, garbage collection, erase control
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays

Abstract

Apparatus and method for data management in a memory, such as but not limited to a flash memory array. In accordance with some embodiments, a plurality of data sets in a memory are identified as having a common data content and different physical addresses in the memory. A selected one of the data sets is marked as valid data and the remaining data sets are marked as stale data responsive to evaluation of at least one variable parameter associated with the physical addresses at which the data sets are respectively stored.

Claims

1. A method comprising identifying a plurality of duplicate data sets in a memory each having a common data content and a different physical address, and, responsive to at least one variable parameter associated with the respective physical addresses, marking a selected one of the plurality of data sets as valid data and the remaining ones of the plurality of data sets as stale data.

2. The method of claim 1, further comprising retaining the selected one of the data sets marked as valid data in the memory at the associated physical address and jettisoning the remaining ones of the sets of data from the memory.

3. The method of claim 1, in which the memory is characterized as a flash memory array.

4. The method of claim 1, in which each of the plurality of data sets have a different logical address.

5. The method of claim 1, in which the at least one variable parameter comprises a total accumulated count of read operations applied to the respective data sets.

6. The method of claim 1, in which the at least one variable parameter comprises a total accumulated count of write operations applied to the respective physical addresses.

7. The method of claim 1, in which the at least one variable parameter comprises a total accumulated count of erasure cycles applied to the respective physical addresses.

8. The method of claim 1, further comprising calculating a hash value for each data set and comparing the hash values to identify the data sets as duplicate data sets.

9. The method of claim 1. further comprising scheduling a garbage collection operation to jettison the stale data from the array.

10. The method of claim 1, in which the plurality of data sets comprise different sectors of data stored in different erasure blocks.

11. An apparatus comprising a memory in which a plurality of data blocks are stored and a deduplication processor configured to identify first and second data blocks in the memory having a common data content and different respective physical addresses, to evaluate at least one variable parameter associated with each of the respective physical addresses, and to select one of the first or second data blocks for retention in the memory and to select the remaining one of the first or second data blocks for jettisoning from the memory responsive to the at least one parameter.

12. The apparatus of claim 11, in which the deduplication processor comprises a hash generator which generates respective first and second hash values responsive to the data content of the first and second data blocks, and a comparison circuit which compares the first and second hash values to identify the first and second data blocks as duplicate blocks.

13. The apparatus of claim 11, in which the deduplication processor further comprises a parametric evaluation module which evaluates a plurality of parameters associated with each of the physical addresses of the first and second blocks to select the respective retained and jettisoned blocks.

14. The apparatus of claim 11, in which the deduplication processor comprises a selection module which selects a population of data blocks for evaluation responsive to access rates associated with the data blocks.

15. The apparatus of claim 11, in which the memory is characterized as a flash memory array comprising a plurality of erasure blocks, the first data block stored in a first erasure block of the flash memory array and the second data block stored in a different, second erasure block of the flash memory array.

16. The apparatus of claim 11, in which the at least one variable parameter comprises age, error rate, accumulated read count or accumulated write/erasure count.

17. A data storage device comprising a main memory and a control circuit adapted to direct data transfer operations between the main memory and a host device, the control circuit further adapted to remove duplicate data sets from the main memory by identifying first and second data sets sharing a common data content and stored at different physical addresses within the main memory, marking a selected one of the first or second data sets as valid data and the remaining one of the first or second data sets as stale data responsive to a variable parameter associated with each of the respective physical addresses, and jettisoning the stale data from the main memory.

18. The data storage device of claim 17, in which the control circuit is further adapted to generate a first hash value from the data content of the first data set and a second hash value from the data content of the second data set, and to compare the first and second hash values.

19. The data storage device of claim 17, in which the main memory is characterized as a flash memory array.

20. The data storage device of claim 17, in which the variable parameter comprises a total accumulated count of erasure operations upon the respective physical addresses of the first and second data sets.