File update tracking

• US 9,430,488 B2
• Filed: 01/30/2015
• Issued: 08/30/2016
• Est. Priority Date: 03/30/2011
• Status: Active Grant

• Current Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Original Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Inventors: Bath, Amritpal Singh, Blank, Mitchell Neuman, Patel, Vishal, Sorkin, Stephen Phillip
• Primary Examiner(s): Le, Hung
• Application Number:
  US14/611,156
• Publication Number:
  US 20150154217A1
• Time in Patent Office: 578 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/174 Redundancy elimination performed by the file system management of the data involved in backup or backup restore using de-duplication of the data G06F11/14
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data

Abstract

Embodiments are directed towards managing and tracking item identification of a plurality of items to determine if an item is a new or existing item, where an existing item has been previously processed. In some embodiments, two or more item identifiers may be generated. In one embodiment, generating the two or more item identifiers may include analyzing the item using a small item size characteristic, a compressed item, or for an identifier collision. The two or more item identifiers may be employed to determine if the item is a new or existing item. In one embodiment, the two or more item identifiers may be compared to a record about an existing item to determine if the item is a new or existing item. If the item is an existing item, then the item may be further processed to determine if the existing item has actually changed.

Claims

1. A method, comprising:

periodically determining a status of each file among a plurality of files in a folder at times corresponding to a time value associated with each file, the time value associated with each file indicating a length of time between status determinations for that file;

based on determining that a file among the plurality of files has been updated, decreasing its associated time value, the time value affecting a next status determination for the file;

based on determining that the file among the plurality of files has not been updated, increasing its associated time value, the time value affecting a next status determination for the file;

detecting that a new file has been added to the plurality of files; and

setting a time value associated with the new file to a default value.

2. The method as recited in claim 1, further comprising:

based on determining that the file among the plurality of files has been updated, analyzing data in the file from a last accessed position to an end of the file.

3. The method as recited in claim 1, further comprising:

based on determining that the file among the plurality of files has been updated, analyzing data in the file from a last accessed position to an end of the file;

updating the last accessed position to the end of the file.

4. The method as recited in claim 1, further comprising:

detecting that a sub-folder exists in the folder;

wherein the determining step periodically determines a status of each file in the sub-folder.

5. The method as recited in claim 1, further comprising:

detecting that a sub-folder exists in the folder;

wherein the determining step periodically determines a status of each sub-folder file among a plurality of sub-folder files in the sub-folder;

based on determining that a sub-folder file among the plurality of sub-folder files in the sub-folder has been updated, decreasing its associated time value, the time value affecting a next status determination for the sub-folder file;

based on determining that the sub-folder file among the plurality of sub-folder files in the sub-folder has not been updated, increasing its associated time value, the time value affecting a next status determination for the sub-folder file.

6. The method as recited in claim 1, further comprising:

detecting that a sub-folder exists in the folder;

wherein the determining step periodically determines a status of each sub-folder file among a plurality of sub-folder files in the sub-folder;

based on determining that a sub-folder file among the plurality of sub-folder files in the sub-folder has been updated, decreasing its associated time value, the time value affecting a next status determination for the sub-folder file;

based on determining that the sub-folder file among the plurality of sub-folder files in the sub-folder has not been updated, increasing its associated time value, the time value affecting a next status determination for the sub-folder file;

based on determining that the sub-folder file among the plurality of sub-folder files has been updated, analyzing data in the sub-folder file from a last accessed position to an end of the sub-folder file.

7. The method as recited in claim 1, further comprising:

detecting that a sub-folder exists in the folder;

wherein the determining step periodically determines a status of each sub-folder file among a plurality of sub-folder files in the sub-folder;

based on determining that a sub-folder file among the plurality of sub-folder files in the sub-folder has been updated, decreasing its associated time value, the time value affecting a next status determination for the sub-folder file;

based on determining that the sub-folder file among the plurality of sub-folder files in the sub-folder has not been updated, increasing its associated time value, the time value affecting a next status determination for the sub-folder file;

based on determining that the sub-folder file among the plurality of sub-folder files has been updated, analyzing data in the sub-folder file from a last accessed position to an end of the sub-folder file;

updating the last accessed position to the end of the sub-folder file.

8. A non-transitory computer readable storage medium, storing software instructions, which when executed by one or more processors cause performance of:

periodically determining a status of each file among a plurality of files in a folder at times corresponding to a time value associated with each file, the time value associated with each file indicating a length of time between status determinations for that file;

based on determining that a file among the plurality of files has been updated, decreasing its associated time value, the time value affecting a next status determination for the file;

based on determining that the file among the plurality of files has not been updated, increasing its associated time value, the time value affecting a next status determination for the file;

detecting that a new file has been added to the plurality of files; and

setting a time value associated with the new file to a default value.

9. The non-transitory computer readable storage medium as recited in claim 8, further comprising:

based on determining that the file among the plurality of files has been updated, analyzing data in the file from a last accessed position to an end of the file.

10. The non-transitory computer readable storage medium as recited in claim 8, further comprising:

based on determining that the file among the plurality of files has been updated, analyzing data in the file from a last accessed position to an end of the file;

updating the last accessed position to the end of the file.

11. The non-transitory computer readable storage medium as recited in claim 8, further comprising:

detecting that a sub-folder exists in the folder;

wherein the determining step periodically determines a status of each file in the sub-folder.

12. The non-transitory computer readable storage medium as recited in claim 8, further comprising:

detecting that a sub-folder exists in the folder;

wherein the determining step periodically determines a status of each sub-folder file among a plurality of sub-folder files in the sub-folder;

based on determining that a sub-folder file among the plurality of sub-folder files in the sub-folder has been updated, decreasing its associated time value, the time value affecting a next status determination for the sub-folder file;

based on determining that the sub-folder file among the plurality of sub-folder files in the sub-folder has not been updated, increasing its associated time value, the time value affecting a next status determination for the sub-folder file.

13. The non-transitory computer readable storage medium as recited in claim 8, further comprising:

detecting that a sub-folder exists in the folder;

wherein the determining step periodically determines a status of each sub-folder file among a plurality of sub-folder files in the sub-folder;

based on determining that a sub-folder file among the plurality of sub-folder files in the sub-folder has been updated, decreasing its associated time value, the time value affecting a next status determination for the sub-folder file;

based on determining that the sub-folder file among the plurality of sub-folder files in the sub-folder has not been updated, increasing its associated time value, the time value affecting a next status determination for the sub-folder file;

based on determining that the sub-folder file among the plurality of sub-folder files has been updated, analyzing data in the sub-folder file from a last accessed position to an end of the sub-folder file.

14. The non-transitory computer readable storage medium as recited in claim 8, further comprising:

detecting that a sub-folder exists in the folder;

wherein the determining step periodically determines a status of each sub-folder file among a plurality of sub-folder files in the sub-folder;

based on determining that a sub-folder file among the plurality of sub-folder files in the sub-folder has been updated, decreasing its associated time value, the time value affecting a next status determination for the sub-folder file;

based on determining that the sub-folder file among the plurality of sub-folder files in the sub-folder has not been updated, increasing its associated time value, the time value affecting a next status determination for the sub-folder file;

based on determining that the sub-folder file among the plurality of sub-folder files has been updated, analyzing data in the sub-folder file from a last accessed position to an end of the sub-folder file;

updating the last accessed position to the end of the sub-folder file.

15. An apparatus, comprising:

a file status determination subsystem, implemented at least partially in hardware, that periodically determines a status of each file among a plurality of files in a folder at times corresponding to a time value associated with each file, the time value associated with each file indicating a length of time between status determinations for that file;

a time value updater, implemented at least partially in hardware, that based on determining that a file among the plurality of files has been updated, decreases its associated time value, the time period affecting a next status determination for the file;

wherein the time value updater, based on determining that the file among the plurality of files has not been updated, increases its associated time value, the time value affecting a next status determination for the file;

a new file detector, implemented at least partially in hardware, that detects that a new file has been added to the plurality of files; and

a time value initializer, implemented at least partially in hardware, that sets a time value associated with the new file to a default value.

16. The apparatus as recited in claim 15, further comprising:

a data analysis subsystem, implemented at least partially in hardware, that based on determining that the file among the plurality of files has been updated, analyzes data in the file from a last accessed position to an end of the file.

17. The apparatus as recited in claim 15, further comprising:

a data analysis subsystem, implemented at least partially in hardware, that based on determining that the file among the plurality of files has been updated, analyzes data in the file from a last accessed position to an end of the file;

a subsystem, implemented at least partially in hardware, that updates the last accessed position to the end of the file.

18. The apparatus as recited in claim 15, further comprising:

a subsystem, implemented at least partially in hardware, that detects that a sub-folder exists in the folder;

wherein the determination subsystem periodically determines a status of each file in the sub-folder.

19. The apparatus as recited in claim 15, further comprising:

a subsystem, implemented at least partially in hardware, that detects that a sub-folder exists in the folder;

wherein the determination subsystem periodically determines a status of each sub-folder file among a plurality of sub-folder files in the sub-folder;

wherein the time period update system, based on determines that a sub-folder file among the plurality of sub-folder files in the sub-folder has been updated, decreases its associated time value, the time value affecting a next status determination for the sub-folder file;

wherein the time period update system, based on determining that the sub-folder file among the plurality of sub-folder files in the sub-folder has not been updated, increases its associated time value, the time value affecting a next status determination for the sub-folder file.

20. The apparatus as recited in claim 15, further comprising:

a subsystem, implemented at least partially in hardware, that detects that a sub-folder exists in the folder;

wherein the determination subsystem periodically determines a status of each sub-folder file among a plurality of sub-folder files in the sub-folder;

wherein the time period update system, based on determining that a sub-folder file among the plurality of sub-folder files in the sub-folder has been updated, decreases its associated time value, the time value affecting a next status determination for the sub-folder file;

wherein the time period update system, based on determining that the sub-folder file among the plurality of sub-folder files in the sub-folder has not been updated, increases its associated time value, the time value affecting a next status determination for the sub-folder file;

a data analysis subsystem, implemented at least partially in hardware, that based on determining that the sub-folder file among the plurality of sub-folder files has been updated, analyzes data in the sub-folder file from a last accessed position to an end of the sub-folder file.

21. The apparatus as recited in claim 15, further comprising:

a subsystem, implemented at least partially in hardware, that detects that a sub-folder exists in the folder;

wherein the determination subsystem periodically determines a status of each sub-folder file among a plurality of sub-folder files in the sub-folder;

wherein the time period update system, based on determining that a sub-folder file among the plurality of sub-folder files in the sub-folder has been updated, decreases its associated time value, the time value affecting a next status determination for the sub-folder file;

wherein the time period update system, based on determining that the sub-folder file among the plurality of sub-folder files in the sub-folder has not been updated, increases its associated time value, the time value affecting a next status determination for the sub-folder file;

a data analysis subsystem, implemented at least partially in hardware, that based on determining that the sub-folder file among the plurality of sub-folder files has been updated, analyzes data in the sub-folder file from a last accessed position to an end of the sub-folder file;

a subsystem, implemented at least partially in hardware, that updates the last accessed position to the end of the sub-folder file.