METHOD AND SYSTEM TO SCAN DATA FROM A SYSTEM THAT SUPPORTS DEDUPLICATION

• US 20110225129A1
• Filed: 03/15/2010
• Published: 09/15/2011
• Est. Priority Date: 03/15/2010
• Status: Active Grant

• Current Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Original Assignee: Symantec Corporation (NortonLifeLock Inc.)
• Inventors: Agrawal, Mukund
• Granted Patent: US 8,832,042 B2
• Time in Patent Office: Days
• US Class Current: 707/692
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641

Abstract

An interface is disclosed that makes information obtained from a file deduplication process available to an application for the efficient operation thereof. A data deduplication repository is scanned to determine a plurality of file segments and respective checksum values associated with the segments. A data structure is generated that allows shared segments to be identified by indexing using a common checksum value. The segments also indicate the file to which they belong and may also include a timestamp value. This data structure is updated as files are modified, etc. The data structure is accessible to an application program so that the application program can readily determine which segments are shared between multiple files. With this information, the application can efficiently process the segment once rather than multiple times. Timestamps can be used by the application to efficiently identify only those segments that were accessed after a given time.

Claims

1. A method of providing file information relating to data deduplication in a computer system, comprising:

accessing a plurality of files, wherein each file comprises a plurality of segments;

accessing a data repository storing data resultant from a file deduplication process including accessing a plurality of checksum values associated with said plurality of segments;

identifying segments of said plurality of files having a same checksum value;

generating a data association structure by associating said segments of said plurality of files having said same checksum value, wherein a first checksum value is operable as an index into said data association structure for obtaining segments having said first checksum value; and

storing said data association structure in a computer memory operable to be accessed by an application program.

2. The method as described in claim 1 wherein within said data association structure, each segment indicates a respective file to which each segment is associated.

3. The method as described by claim 1 further comprising:

accessing said data repository to identify a new checksum value associated with an altered segment of said plurality of segments;

updating said data association structure based on said new checksum value to form an updated data association structure; and

storing said updated data association structure.

4. The method as described by claim 1 further comprising:

from said data repository, accessing a plurality of respective time stamps associated with said plurality of segments wherein each respective time stamp indicates a last time an associated segment was altered.

5. The method as described by claim 4, wherein said data association structure further comprises said plurality of time stamps associated with said plurality of checksum values.

6. The method as described by claim 5 further comprising:

from said data repository, accessing a new time stamp associated with an altered segment of said plurality of segments;

updating said data association structure to form an updated data association structure; and

storing said updated data association structure.

7. The method as described by claim 1 further comprising:

providing a portion of said data association structure to an application requesting access to a segment of said plurality of segments corresponding to said data association structure.

8. The method as described by claim 1 further comprising:

accessing a new file comprising a plurality of segments;

from said data repository, accessing a plurality of respective checksum values associated with said plurality of segments of said new file;

identifying segments of said new file and segments of said plurality of files having a same checksum value;

associating segments having a same checksum value to one another based on said identifying segments of said new file and segments of said plurality of files having a same checksum value to form an updated data association structure; and

storing said updated data association structure.

9. A computer readable storage medium having stored thereon, computer executable instructions that, if executed by a computer system cause the computer system to perform a method of providing file information relating to data deduplication, said method comprising:

accessing a plurality of files, wherein each file comprises a plurality of segments;

accessing a data repository storing data resultant from a file deduplication process including accessing a plurality of checksum values associated with said plurality of segments;

identifying segments of said plurality of files having a same checksum value;

generating a data association structure by associating said segments of said plurality of files having said same checksum value, wherein a first checksum value is operable as an index into said data association structure for obtaining segments having said first checksum value, and wherein, within said data association structure, each segment indicates a respective file to which each segment is associated; and

storing said data association structure in computer memory operable to be accessed by an application program.

10. The computer readable storage medium as described by claim 9 wherein said method further comprises:

accessing said data repository to identify a new checksum value associated with an altered segment of said plurality of segments;

updating said data association structure based on said new checksum value to form an updated data association structure; and

storing said updated data association structure.

11. The computer readable storage medium as described by claim 9 wherein said method further comprises:

from said data repository, accessing a plurality of respective time stamps associated with said plurality of segments wherein each respective time stamp indicates a last time an associated segment was altered.

12. The computer readable storage medium as described by claim 11, wherein said data association structure further comprises said plurality of time stamps associated with said plurality of checksum values.

13. The computer readable storage medium as described by claim 12 wherein said method further comprises:

from said data repository, accessing a new time stamp associated with an altered segment of said plurality of segments;

updating said data association structure to form an updated data association structure; and

storing said updated data association structure.

14. The computer readable storage medium as described by claim 9 wherein said method further comprises:

providing a portion of said data association structure to an application requesting access to a segment of said plurality of segments corresponding to said data association structure.

15. The computer readable storage medium as described by claim 9 wherein said method further comprises:

accessing a new file comprising a plurality of segments;

from said data repository, accessing a plurality of respective checksum values associated with said plurality of segments of said new file;

identifying segments of said new file and segments of said plurality of files having a same checksum value;

associating segments having a same checksum value to one another based on said identifying segments of said new file and segments of said plurality of files having a same checksum value to form an updated data association structure; and

storing said updated data association structure.

16. A method of providing information relating to a deduplication process in a computer system, said method comprising:

generating a segment index data structure responsive to performing a data deduplication process on a plurality of files of a file system, wherein each file of said plurality of files comprises a plurality of segments and wherein said segment index data structure comprises a listing of unique segments within said plurality of segments;

receiving a request from a requesting application for segment information;

responsive to said request, scanning said segment index data structure; and

responsive to said scanning, supplying segment information to said requesting application.

17. A method as described in claim 16 wherein said request includes an application timestamp indicating a last time said requesting application processed data within said file system and wherein further said scanning comprises scanning only respective segments of said segment index data structure having respective timestamps that are after said application timestamp.

18. A method as described in claim 16 wherein said segment information comprises, for each respective segment, a segment identifier and a list of files to which said respective segment belongs and further comprising said requesting application using said segment information to determine whether or not to process respective segments.

19. A method as described in claim 16 wherein said scanning and said supplying are performed iteratively.

20. A method as described in claim 16 wherein said unique segments are identified by respective unique checksum values and wherein further said segment index data structure resides within a repository unit of a data deduplication unit.