Object synthesis

• US 9,633,032 B2
• Filed: 04/30/2012
• Issued: 04/25/2017
• Est. Priority Date: 04/30/2012
• Status: Expired due to Fees

• Current Assignee: Quantum Corporation (Chi Ko Investment Co., Ltd.)
• Original Assignee: Quantum Corporation (Chi Ko Investment Co., Ltd.)
• Inventors: Stoakes, Timothy, Leppard, Andrew
• Primary Examiner(s): Saeed, Usmaan
• Assistant Examiner(s): VO, CECILE H
• Application Number:
  US13/459,295
• Publication Number:
  US 20130290275A1
• Time in Patent Office: 1,821 Days
• Field of Search: 707 99204, 7079992, 707564, 707640, 707644, 707646, 707647, 707696, 707698, 707693, 707E17005, 707692, 704258
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0673 Single storage device

Abstract

Apparatus, methods, and other embodiments associated with object synthesis are described. One example apparatus includes logic for identifying a block in a data de-duplication repository and for identifying a reference to the block. The apparatus also includes logic for representing a source object using a first named, organized collection of references to blocks in the data de-duplication repository and logic for representing a target object using a second named, organized collection of references. The apparatus is configured to synthesize the target object from the source object. Since synthesis may be complicated by edge cases, the apparatus is configured to account for conditions including a block in the target object needing less than all the data in a source object block, data to be used to synthesize the target object residing in a sparse hole in a data stream, and the target object needing data not present in the source object.

Claims

1. A non-transitory computer-readable medium storing computer-executable instructions that when executed by a computer cause the computer to perform a method, the method comprising:

controlling the computer to synthesize a target entity from a source entity,

where the source entity comprises one or more blocks processed by a data de-duplication process and stored in a block repository, where the data de-duplication process is a post-processing data de-duplication process, a storage layout aware data de-duplication process, a storage layout aware cloning process, a storage layout aware replication process, a snapshot/replay process, or a de-duplication defragmentation process,

where a representation of the source entity comprises a first named, organized collection of references to blocks produced by the data de-duplication process,

where a representation of the target entity is a second named, organized collection of references to blocks produced by the data de-duplication process, and

where synthesizing the target entity from the source entity comprises;

manipulating the representation of the target entity by copying a reference from the representation of the source entity to the representation of the target entity;

establishing the representation of the target entity by adding to the representation of the target entity zero or more references not found in the representation of the source entity;

establishing the representation of the target entity by adding to the representation of the target entity at least one reference not found in the representation of the source entity;

or

logically adding to the target entity a block not referred to by the representation of the source entity; and

selectively manipulating a parse context in use by the data de-duplication process to a first default state, a second state associated with a parse context in use at the time the de-duplication process processed a block associated with the source entity, or a third state that emulates a parse context present before a discontinuity in a target stream, where the parse context comprises a current position in an incident data stream, a rolling hash value, a pre-determined condition being evaluated by the de-duplication process, a pattern being sought by the de-duplication process, or progress towards the pattern being sought.

2. The non-transitory computer-readable medium of claim 1, where the representation of the source entity includes a first list of pointers to blocks stored in the block repository and where the representation of the target entity includes a second list of pointers to blocks stored in the block repository.

3. The non-transitory computer-readable medium of claim 1, the method comprising, upon determining that a source element referred to by a reference to be copied from the representation of the source entity to the representation of the target entity includes data that is not needed in a target element, controlling the computer to create a new block from just the data needed from the source element to synthesize the target element and controlling the computer to provide a reference to the new block for use in synthesizing the target entity.

4. The non-transitory computer-readable medium of claim 3, the method comprising selectively controlling the computer to reparse just the data needed from the source element to synthesize the target element, or controlling the computer to physically mask or logically mask the data that is not needed.

5. The non-transitory computer-readable medium of claim 4, where selectively controlling the computer to reparse just the data needed from the source element is determined as a function of an ingest speed associated with the data de-duplication process, a storage efficiency read-back speed associated with the data de-duplication process, a stability of data being processed by the data de-duplication process, a shape of input/output, resource constraints available during a defined period of time, or resource constraints present at the time of processing.

6. The non-transitory computer-readable medium of claim 5, the method comprising:

upon determining that a source element referred to by a reference to be copied from the representation of the source entity to the representation of the target entity includes data that is not needed to synthesize the target element,

controlling the computer to annotate a target reference with information sufficient to logically remove the data from the source element from the point of view of the target element, where the information sufficient to logically remove the data includes start/end information or include/exclude information.

7. The non-transitory computer-readable medium of claim 6, the method comprising:

upon determining that a source location in a source stream from which data used to synthesize the target entity will be copied is located in a sparse hole, synthesizing the target element so that a sparse hole associated with the target entity will be smaller than the sparse hole in the source stream, or synthesizing the target element so that a sparse hole associated with the target entity will be larger than the sparse hole in the source stream.

8. The non-transitory computer-readable medium of claim 7, the method comprising:

upon determining that a block in the block repository was involved in synthesizing the target element, selectively managing a life cycle attribute associated with the block.

9. An apparatus, comprising:

a processor;

a memory;

a set of logics; and

an interface to connect the processor, the memory, and the set of logics, the set of logics comprising;

a first logic that processes a source object and that processes a first organized set of pointers to blocks associated with the source object, where the blocks are stored in a data de-duplication block data store, where the blocks were processed by a de-duplication process, and where the organized set of pointers facilitates recreating the source object from blocks stored in the data de-duplication block data store, where the de-duplication process is a post-processing data de-duplication process, a storage layout aware data de-duplication process, a storage layout aware cloning process, a storage layout aware replication process, a snapshot/replay process, or a de-duplication defragmentation process;

a second logic that processes a target object and that processes a second organized set of pointers to blocks associated with the target object, where the second organized set of pointers facilitates recreating the target object from blocks stored in the data de-duplication block data store; and

a third logic that synthesizes the target object from the source object by controlling the first logic to selectively provide members of the first set of pointers to the second logic and by controlling the second logic to arrange the pointers provided from the first set of pointers in the second set of pointers,

where the third logic provides a new pointer to the second logic, where the new pointer is not provided by the first logic, and where the new pointer is associated with new data processed by the de-duplication process, and where the new pointer is provided in response to determining that the target object cannot be synthesized completely from data associated only with the source object; and

where the third logic selectively manipulates a parse context associated with the de-duplication process in response to determining that the target object cannot be synthesized completely from data associated with the source object, where the parse context includes a current position in an incident data stream, a rolling hash value, a pre-determined condition being evaluated by the de-duplication process, a pattern being sought by the de-duplication process, or progress towards the pattern being sought.

10. The apparatus of claim 9, where the third logic reacts to a condition where a target block needs less than all the data in a block pointed to by a member of the first set of pointers by producing a transformed block and providing a pointer to the transformed block to the second logic.

11. The apparatus of claim 10, where the third logic produces the transformed block by performing one of:

creating a new block that includes just the data needed by the target block from the source block; and

manipulating the data that is not needed in the source block to make that data unavailable in the target block, where manipulating the data that is not needed includes physical or logical masking of data in the source object.

12. The apparatus of claim 9, where the third logic reacts to a condition where data to be used to synthesize the target object resides in a sparse hole in a data stream by manipulating a block associated with the target object to manipulate a size of a corresponding sparse hole associated with the target object.

13. The apparatus of claim 9, where the third logic manages a life cycle attribute of a block involved in the synthesis of the second object.

14. A system, comprising:

means for identifying a block in a data de-duplication repository;

means for identifying a reference to a block in a data de-duplication repository;

means for representing a source object using a first named, organized collection of references to blocks in the data de-duplication repository;

means for representing a target object using a second named, organized collection of references to blocks in the data de-duplication repository,

means for synthesizing the target object from the source object by manipulating the second named organized collection of references based, at least in part, on data available through the first named organized collection of references, where the means for synthesizing is configured to account for a plurality of conditions, where the plurality of conditions includes;

a condition where a block associated with the target object needs less than all the data in a block associated with the source object,

a condition where data to be used to synthesize the target object resides in a sparse hole in a data stream; and

a condition where data to be used to synthesize the target object is not present in the source object; and

means for selectively manipulating a parse context associated with a de-duplication process in response to determining that the target object cannot be synthesized completely from data associated with the source object, where the parse context comprises a current position in an incident data stream, a rolling hash value, a pre-determined condition being evaluated by the de-duplication process, a pattern being sought by the de-duplication process, or progress towards the pattern being sought.