Object deduplication and application aware snapshots

• US 9,251,161 B2
• Filed: 04/04/2014
• Issued: 02/02/2016
• Est. Priority Date: 10/07/2008
• Status: Active Grant

• Current Assignee: Quest Software, Inc.
• Original Assignee: Dell Products LP (Dell Technologies Inc.)
• Inventors: Rao, Goutham, Brueggemann, Eric, George, Carter
• Primary Examiner(s): Wong, Leslie
• Application Number:
  US14/245,084
• Publication Number:
  US 20140222769A1
• Time in Patent Office: 669 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/183 Provision of network file services by network file servers, e.g. by using NFS, CIFS network file access protocols H04L67/1097
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 40/205 Parsing

Abstract

Embodiments deploy delayering techniques, and the relationships between successive versions of a rich-media file become apparent. With this, modified rich-media files suddenly present far smaller storage overhead as compared to traditional application-unaware snapshot and versioning implementations. Optimized file data is stored in suitcases. As a file is versioned, each new version of the file is placed in the same suitcase as the previous version, allowing embodiments to employ correlation techniques to enhance optimization savings.

Claims

1. A method comprising:

parsing a file to identify boundaries for a plurality of first level objects included in the file in order to determine whether each of the plurality of first level objects is compound or primitive, wherein a first compound object included in the file encapsulates a plurality of second level objects and a primitive object is the most basic representation of a discrete data structure in the file;

recursively parsing the first compound object to identify boundaries for a plurality of second level objects included in the first compound object;

determining whether each of the plurality of second level objects is compound or primitive;

identifying a plurality of lowest level primitive objects, wherein the plurality of lowest level primitive objects are basic representations of discrete data structures, wherein metadata for each of the plurality of lowest level primitive objects is stored redundantly in a suitcase file, and wherein deduplication boundaries are set at boundaries of the plurality of lowest level primitive objects;

decompressing the plurality of lowest level primitive objects;

recompressing the plurality of lowest level primitive objects with a plurality of object specific optimization algorithms.

2. The method of claim 1, wherein object based deduplication is applied to the plurality of lowest level primitive objects.

3. The method of claim 1, wherein compound object include zip archives and PDF files.

4. The method of claim 1, wherein primitive objects include JPEG images, TIFF images, and CAD drawings.

5. The method of claim 1, wherein deduplication is applied to the plurality of lowest level primitive objects.

6. The method of claim 1, wherein index information is stored in the suitcase file to allow access to each of the plurality of lowest level primitive objects.

7. The method of claim 6, wherein an index number in a file name is used to access index information in the suitcase file to allow access to corresponding compressed data in the suitcase file.

8. A non-transitory computer readable medium comprising:

computer code for parsing a file to identify boundaries for a plurality of first level objects included in the file in order to determine

whether each of the plurality of first level objects is compound or primitive, wherein a first compound object included in the file encapsulates a plurality of second level objects and a primitive object is the most basic representation of a discrete data structure in the file;

computer code for recursively parsing the first compound object to identify boundaries for a plurality of second level objects included in the first compound object;

computer code for determining whether each of the plurality of second level objects is compound or primitive;

computer code for identifying a plurality of lowest level primitive objects, wherein the plurality of lowest level primitive objects are basic representations of discrete data structures, wherein metadata for each of the plurality of lowest level primitive objects is stored redundantly in a suitcase file, and wherein deduplication boundaries are set at boundaries of the plurality of lowest level primitive objects;

computer code for decompressing the plurality of lowest level primitive objects;

computer code for recompressing the plurality of lowest level primitive objects with a plurality of object specific optimization algorithms.

9. The non-transitory computer readable medium of claim 8, wherein object based deduplication is applied to the plurality of lowest level primitive objects.

10. The non-transitory computer readable medium of claim 8, wherein compound object include zip archives and PDF files.

11. The non-transitory computer readable medium of claim 8, wherein primitive objects include JPEG images, TIFF images, and CAD drawings.

12. The non-transitory computer readable medium of claim 8, wherein deduplication is applied to the plurality of lowest level primitive objects.

13. The non-transitory computer readable medium of claim 8, wherein index information is stored in the suitcase file to allow access to each of the plurality of lowest level primitive objects.

14. The non-transitory computer readable medium of claim 13, wherein an index number in a file name is used to access index information in the suitcase file to allow access to corresponding compressed data in the suitcase file.

15. A system comprising:

an interface operable to receive a file comprising a plurality of first level objects;

a parser operable to recursively parse the file to identify a plurality of first level objects, second level objects, and lowest level objects included in the file;

a processor configured to determine through parsing the file whether each of the plurality of first level objects is compound or primitive, wherein a first compound object included in the file encapsulates a plurality of second level objects and a primitive object is the most basic representation of a discrete data structure in the file, the processor further configured to determine whether each of the plurality of second level objects is compound or primitive, wherein the plurality of lowest level primitive objects are identified, wherein the plurality of lowest level primitive objects are basic representations of discrete data structures, wherein metadata for each of the plurality of lowest level primitive objects is stored redundantly in a suitcase file, and wherein deduplication boundaries are set at boundaries of the plurality of lowest level primitive objects;

a decompression mechanisms configured to decompress the plurality of lowest level primitive objects, wherein the plurality of lowest level primitive objects are recompressed with a plurality of object specific optimization algorithms.

16. The system of claim 15, wherein object based deduplication is applied to the plurality of lowest level primitive objects.