Adaptive experience based De-duplication

• US 20130151483A1
• Filed: 12/07/2011
• Published: 06/13/2013
• Est. Priority Date: 12/07/2011
• Status: Abandoned Application

• Current Assignee: Quantum Corporation (Chi Ko Investment Co., Ltd.)
• Original Assignee: Quantum Corporation (Chi Ko Investment Co., Ltd.)
• Inventors: Tofano, Jeffrey
• Application Number:
  US13/373,990
• Publication Number:
  US 20130151483A1
• Time in Patent Office: Days
• US Class Current: 707/692
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0671 In-line storage system
• CPC Class Codes: G06F 7/00 Methods or arrangements for processing data by operating upon the order or content of the data handled logic circuits H03K19/00

Abstract

Example apparatus and methods associated with adaptive experience based de-duplication are provided. One example data de-duplication apparatus includes a de-duplication logic, an experience logic, and a reconfiguration logic. The de-duplication logic may be configured to perform data de-duplication according to a configurable approach that is a function of a pre-defined constraint. The experience logic may be configured to acquire de-duplication performance experience data. The reconfiguration logic may be configured to selectively reconfigure the configurable approach on the apparatus as a function of the de-duplication performance experience data. In different examples, dynamic reconfiguration may be performed locally and/or in a distributed manner based on local and/or distributed data that is acquired on a per actor (e.g., user, application) basis and/or on a per entity (e.g., computer, data stream) basis.

Claims

1. A method, comprising:

accessing de-duplication experience data; and

selectively automatically and dynamically reconfiguring computerized de-duplication as a function of the de-duplication experience data.

2. The method of claim 1, where the de-duplication experience data comprises one or more of, performance time data, and data reduction data.

3. The method of claim 2, where reconfiguring computerized de-duplication comprises changing one or more of, a boundary placing approach, a chunking approach, a hashing approach, a sampling approach, a desired mean chunk length, and a uniqueness determination approach.

4. The method of claim 1, comprising reconfiguring local computerized de-duplication based on local de-duplication experience data.

5. The method of claim 1, comprising reconfiguring distributed computerized de-duplication based on distributed de-duplication experience data.

6. The method of claim 1, comprising processing de-duplication reconstitution information for an item that has been de-duplicated using a reconfigured computerized de-duplication, where the de-duplication reconstitution information identifies one or more of, the computerized de-duplication employed to de-duplicate the item, and the de-duplication experience data employed to reconfigure the computerized de-duplication.

7. The method of claim 6, where processing the de-duplication reconstitution information comprises adding de-duplication reconstitution information to a de-duplication data structure, where the information identifies one or more of, the computerized de-duplication employed to de-duplicate the item, and the de-duplication experience data employed to reconfigure the computerized de-duplication.

8. The method of claim 6, where processing the de-duplication reconstitution information comprises associating de-duplication reconstitution information with a data source, where the de-duplication reconstitution information identifies one or more of, the computerized de-duplication approach employed to de-duplicate data from the data source, and the de-duplication experience data employed to reconfigure the computerized de-duplication for the data source.

9. The method of claim 6, where processing the de-duplication reconstitution information comprises adding de-duplication reconstitution information to the item that has been de-duplicated using a reconfigured computerized de-duplication.

10. The method of claim 1, comprising:

performing two or more reconfigurable computerized de-duplication approaches in parallel;

acquiring performance experience data for the two or more reconfigurable approaches; and

selectively automatically and dynamically reconfiguring the computerized de-duplication to perform one of the two or more reconfigurable computerized de-duplication approaches based on the performance experience data for the two or more reconfigurable approaches.

11. The method of claim 1, comprising dynamically reconfiguring computerized de-duplication as a function of de-duplication experience data on one or more of, a per actor basis, and a per entity basis.

12. A data de-duplication apparatus, comprising:

a processor;

a memory;

a set of logics; and

an interface configured to connect the processor, the memory, and the set of logics,

the set of logics comprising;

a de-duplication logic configured to perform data de-duplication according to a configurable approach, where the configurable approach is a function of a pre-defined constraint;

an experience logic configured to acquire de-duplication performance experience data; and

a reconfiguration logic configured to selectively reconfigure the configurable approach on the apparatus as a function of the de-duplication performance experience data.

13. The apparatus of claim 12, where the de-duplication performance experience data comprises one or more of, data reduction amount data, and data reduction time data.

14. The apparatus of claim 13, where the de-duplication performance experience data comprises one or more of, data from the data de-duplication apparatus, and data from a second, different data de-duplication apparatus.

15. The apparatus of claim 12, where the configurable approach is configurable on one or more of, boundary placement approach, chunking approach, desired mean chunk length, sampling locations, sampling size, uniqueness determination approach, and hashing approach.

16. The apparatus of claim 12, where the reconfiguration logic is configured to selectively reconfigure the configurable approach on a second data de-duplication apparatus as a function of the de-duplication performance experience data.

17. The apparatus of claim 12, where the pre-defined constraint is controlled by the data de-duplication apparatus.

18. The apparatus of claim 12, where the pre-defined constraint is controlled by an entity external to the apparatus.

19. The apparatus of claim 12, where the experience logic is configured to acquire the de-duplication performance experience data on one or more of, a per user basis, and a per entity basis.

20. The apparatus of claim 19, where the reconfiguration logic is configured to selectively reconfigure the configurable approach on one or more of, a per user basis, and a per entity basis.

21. A computer-readable medium storing computer-executable instructions that when executed by a data de-duplication apparatus control the data de-duplication apparatus to perform a method, the method comprising:

acquiring de-duplication experience data comprising performance time data and reduction amount data, where the performance experience data is acquired on one or more of, a per actor basis, and a per entity basis, and where the performance experience data is acquired on one or more of, a local basis, and a distributed basis,

and;

selectively automatically and dynamically changing one or more of, a boundary placing approach, a chunking approach, a hashing approach, a sampling approach, and a uniqueness determination approach for a computerized de-duplication apparatus as a function of the de-duplication experience data,

where the reconfiguring comprises one or more of, reconfiguring local computerized de-duplication based on local de-duplication experience data and reconfiguring distributed computerized de-duplication based on distributed de-duplication experience data, and

where the reconfiguring comprises one or more of, reconfiguring on a per actor basis, and reconfiguring on a per entity basis.

22. A system, comprising:

means for dynamically reconfiguring a computerized de-duplication apparatus based on one or more of, de-duplication time data, and de-duplication reduction data.