OBJECT DEDUPLICATION AND APPLICATION AWARE SNAPSHOTS

• US 20100088277A1
• Filed: 10/07/2009
• Published: 04/08/2010
• Est. Priority Date: 10/07/2008
• Status: Active Grant

• Current Assignee: Quest Software, Inc.
• Original Assignee: Ocarina Networks, Inc. (Dell Technologies Inc.)
• Inventors: Brueggemann, Eric, Rao, Goutham, George, Carter
• Granted Patent: US 8,694,466 B2
• Time in Patent Office: Days
• US Class Current: 707/637
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/183 Provision of network file services by network file servers, e.g. by using NFS, CIFS network file access protocols H04L67/1097
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 40/205 Parsing

Abstract

Embodiments deploy delayering techniques, and the relationships between successive versions of a rich-media file become apparent. With this, modified rich-media files suddenly present far smaller storage overhead as compared to traditional application-unaware snapshot and versioning implementations. Optimized file data is stored in suitcases. As a file is versioned, each new version of the file is placed in the same suitcase as the previous version, allowing embodiments to employ correlation techniques to enhance optimization savings.

Claims

1. A method of reducing data duplication in a computer system storing files, the method comprising:

(a) parsing a file to determine native application boundaries of applications within the file;

(b) determining if the applications are compressed;

(c) decompressing the applications within the file with a technique appropriate for each application;

(d) recursively performing steps (a)-(d) until all primitive objects are uncovered and decompressed;

(e) correlating primitive objects within each file and across a plurality of files accessible by the computer system; and

(f) reducing redundancy between two or more primitive objects within related or versioned files.

2. The method of claim 1, wherein reducing redundancy comprises creating a suitcase file operable to hold metadata and compressed data associated with a plurality of files on a network attached storage system.

3. The method of claim 2, wherein reducing redundancy further comprises storing metadata for each of the plurality of files redundantly in the suitcase file, wherein commit bits are stored with the metadata.

4. The method of claim 3, wherein reducing redundancy further comprises writing compressed data corresponding to each of the plurality files in the suitcase file on the network attached storage system.

5. The method of claim 4, wherein reducing redundancy further comprises storing index information in the suitcase file, the index information allowing access to each of the plurality of files in the suitcase file.

6. The method of claim 2, wherein an index number in a file name is used to access index information in the suitcase file to allow access to corresponding compressed data in the suitcase file.

7. The method of claim 2, wherein the metadata is stored twice and surrounded with commit bits.

8. The method of claim 2, wherein index information comprises a next_index and a next_offset.

9. The method of claim 2, wherein index information further comprises data_length and data_info fields for each of the plurality of files.

10. The method of claim 2, wherein file name index number are included in each of the hard links to the plurality of files in the suitcase file.

11. The method of claim 2, wherein a simulated link is mapped to each of the hard links.

12. The method of claim 11, wherein the simulated link to a compressed file has a name matching the name of the file in precompressed form.

13. A method of managing a data storage system, comprising:

determining if a file is of a compound nature containing underlying primitive objects or of a primitive nature;

extracting the underlying primitive objects from files determined to be compound, at least one extraction comprising two layers of decoding with two different codecs;

correlating the extracted objects with the information they represent; and

reducing information redundancy between two or more primitive objects within related or versioned files.