Incremental scan

• US 9,116,915 B1
• Filed: 03/29/2012
• Issued: 08/25/2015
• Est. Priority Date: 03/29/2012
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Tiwari, Priyank, Lakshman, Nagaraja Mandya
• Primary Examiner(s): Corrielus, Jean M
• Application Number:
  US13/434,348
• Time in Patent Office: 1,244 Days
• Field of Search: 707/201, 707/609, 707/694, 707/652, 707/828, 707/711, 707/805, 709/217, 711/13, 726/24
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/188 Virtual file systems

Abstract

This disclosure relates to a method, article of manufacture, and apparatus for scanning data. In some embodiments, this includes examining attributes of a first directory, wherein the attributes include at least one from a group comprising a next tentative scan date, a modified time, and a dirty indicator, determining if the dirty indicator has been set, if the dirty has been set, scanning files in the first directory, if the dirty indicator has not been set, determining if the first directory has been modified, if the first directory has not been modified, determining if the next tentative scan date equals to a system date, if the next tentative scan date equals to the system date, scanning the files, if the next tentative scan date does not equal to the system date, examining attributes of a second directory, and if the first directory has been modified, scanning the files.

Claims

1. A method for incrementally scanning data, comprising:

defining, for a first directory containing files, an extended attribute structure having a plurality of fields storing attributes related to modification of the files;

after a previous scan to set attributes in the extended attribute structure including a next tentative scan date, a modified time, and a dirty indicator indicating an exception happened during the previous scan and providing a workaround for unexpected behavior of the previous scan, examining during the incremental scan the attributes of the first directory;

determining if the dirty indicator has been set, the dirty indicator contained in a field of the extended attribute structure;

if the dirty indicator has been set, scanning the files in the first directory;

if the dirty indicator has not been set, determining if the first directory has been modified;

if the first directory has not been modified, determining if the next tentative scan date equals to a system date;

if the next tentative scan date equals to the system date, scanning the files;

if the next tentative scan date does not equal to the system date, examining attributes of a second directory; and

if the first directory has been modified, scanning the files in the first directory.

2. The method as recited in claim 1, wherein scanning the files includes

opening a first file of the files;

reading file attributes of the first file from the extended attribute structure;

evaluating the file attributes based on at least one policy;

closing the first file;

determining if an exception has taken place;

if the exception has taken place, setting the dirty indicator;

if no exception has taken place, performing a migration of the first file based on the at least one policy;

comparing the next tentative scan date with a scan date of the first file; and

in the event that the next tentative scan date is not as recent as the scan date of the first file, setting the next tentative scan date to the scan date of the first file.

3. The method as recited in claim 2, wherein opening a first file of the files includes

determining if an end of the first directory has been reached;

based on the determination that the end of the first directory has been reached, updating the attributes;

based on the determination that the end of the directory has not been reached, opening a second file of the files;

reading file attributes of the second file;

evaluating the file attributes based on a policy;

closing the second file;

determining if an exception has taken place;

if the exception has taken place, setting the dirty indicator;

if no exception has taken place, performing a migration of the second file based on the policy; and

calculating the next tentative scan date.

4. The method as recited in claim 3, wherein updating the attributes includes

storing the next tentative scan date in another field of the extended attribute structure;

and

storing a system directory modified time to the modified time in yet another field of the extended attribute structure.

5. The method as recited in claim 2, wherein the at least one policy includes at least one file age-based policy.

6. The method of claim 2, wherein the previous scan is a full scan of the files to set the attributes.

7. The method as recited in claim 1, wherein determining if the first directory has been modified includes

comparing the modified time with a system directory modified time;

in the event that the modified time is not the same as the system directory modified time, indicating the first directory has been modified; and

in the event that the modified time is the same as the system directory modified time, indicating the first directory has not been modified.

8. The method of claim 1 wherein the extended attribute structure comprises a plurality of fields storing respective attributes related to a time or status of modification of at least some of the files in the first directory.

9. The method of claim 8 wherein the extended attribute structure of the first directory is checked prior to performing any file operations on the files to determine whether the first directory contains files that need to be scanned.

10. A system for incrementally scanning data, comprising:

a processor configured to

define, for a first directory containing files, an extended attribute structure having a plurality of fields storing attributes related to modification of the files;

after a previous scan to set attributes in the extended attribute structure including a next tentative scan date, a modified time, and a dirty indicator indicating an exception happened during the previous scan and providing a workaround for unexpected behavior of the previous scan, examine during the incremental scan the attributes of the first directory;

determine if the dirty indicator has been set, the dirty indicator contained in a field of the extended attribute structure;

if the dirty indicator has been set, scan the files in the first directory;

if the dirty indicator has not been set, determine if the first directory has been modified;

if the first directory has not been modified, determine if the next tentative scan date equals to a system date;

if the next tentative scan date equals to the system date, scan the files;

if the next tentative scan date does not equal to the system date, examine attributes of a second directory; and

if the first directory has been modified, scanning the files in the first directory.

11. A computer program product for incrementally scanning data, comprising a non-transitory computer usable medium having machine readable code embodied therein for:

defining, for a first directory containing files, an extended attribute structure having a plurality of fields storing attributes related to modification of the files;

after a previous scan to set attributes in the extended attribute structure including a next tentative scan date, a modified time, and a dirty indicator indicating an exception happened during the previous scan and providing a workaround for unexpected behavior of the previous scan, examining during the incremental scan the attributes of the first directory;

determining if the dirty indicator has been set, the dirty indicator contained in a field of the extended attribute structure;

if the dirty indicator has been set, scanning the files in the first directory;

if the dirty indicator has not been set, determining if the first directory has been modified;

if the first directory has not been modified, determining if the next tentative scan date equals to a system date;

if the next tentative scan date equals to the system date, scanning the files;

if the next tentative scan date does not equal to the system date, examining attributes of a second directory; and

if the first directory has been modified, scanning the files in the first directory.