Object Synthesis

• US 20130290275A1
• Filed: 04/30/2012
• Published: 10/31/2013
• Est. Priority Date: 04/30/2012
• Status: Active Grant

• Current Assignee: Quantum Corporation (Chi Ko Investment Co., Ltd.)
• Original Assignee: Quantum Corporation (Chi Ko Investment Co., Ltd.)
• Inventors: STOAKES, Timothy, Leppard, Andrew
• Granted Patent: US 9,633,032 B2
• Time in Patent Office: Days
• US Class Current: 707/692
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0673 Single storage device

Abstract

Apparatus, methods, and other embodiments associated with object synthesis are described. One example apparatus includes logic for identifying a block in a data de-duplication repository and for identifying a reference to the block. The apparatus also includes logic for representing a source object using a first named, organized collection of references to blocks in the data de-duplication repository and logic for representing a target object using a second named, organized collection of references. The apparatus is configured to synthesize the target object from the source object. Since synthesis may be complicated by edge cases, the apparatus is configured to account for conditions including a block in the target object needing less than all the data in a source object block, data to be used to synthesize the target object residing in a sparse hole in a data stream, and the target object needing data not present in the source object.

Claims

1. A non-transitory computer-readable medium storing computer-executable instructions that when executed by a computer cause the computer to perform a method, the method comprising:

controlling the computer to synthesize a target entity from a source entity,

where the source entity comprises one or more blocks processed by a data de-duplication process and stored in a block repository,

where a representation of the source entity comprises a first named, organized collection of references to blocks produced by the data de-duplication process,

where a representation of the target entity is a second named, organized collection of references to blocks produced by the data de-duplication process, and

where synthesizing the target entity from the source entity comprises manipulating the representation of the target entity by copying one or more references from the representation of the source entity to the representation of the target entity.

2. The non-transitory computer-readable medium of claim 1, where the representation of the source entity includes a first list of pointers to blocks stored in the block repository and where the representation of the target entity includes a second list of pointers to blocks stored in the block repository.

3. The non-transitory computer-readable medium of claim 1, the method comprising, upon determining that a source element referred to by a reference to be copied from the representation of the source entity to the representation of the target entity includes data that is not needed in a target element, controlling the computer to create a new block from just the data needed from the source element to synthesize the target element and controlling the computer to provide a reference to the new block for use in synthesizing the target entity.

4. The non-transitory computer-readable medium of claim 3, the method comprising selectively controlling the computer to perform one of:

storing the new block in the block repository; and

reparsing just the data needed from the source element to synthesize the target element.

5. The non-transitory computer-readable medium of claim 4, where selectively controlling the computer to perform one of storing the new block or reparsing just the data needed from the source element is determined as a function of one or more of, an ingest speed associated with the data de-duplication process, a storage efficiency read-back speed associated with the data de-duplication process, a type of data being processed by the data de-duplication process, a stability of data being processed by the data de-duplication process, a shape of input/output, on resource constraints available during a defined period of time, and on resource constraints present at the time of processing.

6. The non-transitory computer-readable medium of claim 1, the method comprising:

upon determining that a source element referred to by a reference to be copied from the representation of the source entity to the representation of the target entity includes data that is not needed to synthesize the target element,

controlling the computer to annotate the target reference with information sufficient to logically remove the data from the source element from the point of view of the target element.

7. The non-transitory computer-readable medium of claim 1, where synthesizing the target entity from the source entity comprises one or more of:

establishing the representation of the target entity by adding to the representation of the target entity zero or more references not found in the representation of the source entity;

establishing the representation of the target entity by adding to the representation of the target entity at least one reference not found in the representation of the source entity, and

logically adding to the target entity a block not referred to by the representation of the source entity.

8. The non-transitory computer-readable medium of claim 7, the method comprising;

selectively manipulating a parse context in use by the data de-duplication process to one of, a first default state, a second state associated with a parse context in use at the time the de-duplication process processed a block associated with the source entity, and a third state that emulates a parse context present before a discontiguity in the target stream.

9. The non-transitory computer-readable medium of claim 1, the method comprising:

upon determining that a source location in a source stream from which data used to synthesize the target entity will be copied is located in a sparse hole, synthesizing the target element so that a sparse hole associated with the target entity will be smaller than the sparse hole in the source stream.

10. The non-transitory computer-readable medium of claim 1, the method comprising:

upon determining that a block in the block repository was involved in synthesizing the target element, selectively managing a life cycle attribute associated with the block.

11. The non-transitory computer-readable medium of claim 1, the method being performed by one of, an inline data de-duplication process, a post-processing data de-duplication process, a storage layout aware data de duplication process, a storage layout aware cloning process, a storage layout aware replication process, a snapshot/replay process, and a de-duplication defragmentation process.

12. An apparatus, comprising:

a processor;

a memory;

a set of logics; and

an interface to connect the processor, the memory, and the set of logics, the set of logics comprising;

a first logic configured to process a source object and configured to process a first organized set of pointers to blocks associated with the source object, where the blocks are stored in a data de-duplication block data store, and where the organized set of pointers facilitates recreating the source object from blocks stored in the data de-duplication block data store;

a second logic configured to process a target object and configured to process a second organized set of pointers to blocks associated with the target object, where the second organized set of pointers facilitates recreating the target object from blocks stored in the data de-duplication block data store; and

a third logic configured to synthesize the target object from the source object by controlling the first logic to selectively provide members of the first set of pointers to the second logic and by controlling the second logic to arrange the pointers provided from the first set of pointers in the second set of pointers.

13. The apparatus of claim 12, where third logic is configured to react to a condition where a target block needs less than all the data in a block pointed to by a member of the first set of pointers by producing a transformed block and providing a pointer to the transformed block to the second logic.

14. The apparatus of claim 13, where the third logic is configured to produce the transformed block by performing one of:

creating a new block that includes just the data needed by the target block from the source block; and

manipulating the data that is not needed in the source block to make that data unavailable in the target block.

15. The apparatus of claim 12, the third logic being configured to react to a condition where data to be used to synthesize the target object resides in a sparse hole in a data stream by manipulating one or more blocks associated with the target object to manipulate a size of a corresponding sparse hole associated with the target object.

16. The apparatus of claim 12, the third logic being configured to provide a new pointer to the second logic, where the new pointer is not provided by the first logic, and where the new pointer is associated with new data processed by the de-duplication process, and where the new pointer is provided in response to determining that the target object cannot be synthesized completely from data associated only with the source object.

17. The apparatus of claim 16, the third logic being configured to selectively manipulate a parse context associated with the de-duplication process in response to determining that the target object cannot be synthesized completely from data associated with the source object.

18. The apparatus of claim 12, the third logic being configured to manage a life cycle attribute of a block involved in the synthesis of the second object.

19. A system, comprising:

means for identifying a block in a data de-duplication repository;

means for identifying a reference to a block in a data de-duplication repository;

means for representing a source object using a first named, organized collection of references to blocks in the data de-duplication repository;

means for representing a target object using a second named, organized collection of references to blocks in the data de-duplication repository, and means for synthesizing the target object from the source object by manipulating the second named organized collection of references based, at least in part, on data available through the first named organized collection of references, where the means for synthesizing is configured to account for a plurality of conditions, where the plurality of conditions includes;

a condition where a block associated with the target object needs less than all the data in a block associated with the source object,

a condition where data to be used to synthesize the target object resides in a sparse hole in a data stream; and

a condition where data to be used to synthesize the target object is not present in the source object.