System and method for fast file tracking and change monitoring

• US 8,548,961 B2
• Filed: 03/30/2011
• Issued: 10/01/2013
• Est. Priority Date: 03/30/2011
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Bath, Amritpal Singh, Blank, Mitchell Neuman Jr., Patel, Vishal, Sorkin, Stephen Phillip
• Primary Examiner(s): Pham, Khanh
• Assistant Examiner(s): Ahluwalia, Navneet K
• Application Number:
  US13/076,263
• Publication Number:
  US 20120254128A1
• Time in Patent Office: 916 Days
• Field of Search: 707688-690
• US Class Current: 707/689
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases

Abstract

Embodiments are directed towards a dynamic change evaluation mechanism, whereby items having a detected possible change are scheduled for re-evaluation for possible changes at a higher frequency than items detected to not have previously changed, while those items detected as not to have changed are dynamically scheduled for re-evaluation based on an evaluation backlog that may be in turn based, in part, on a time from when an item is assigned an expiration time to when the item is evaluated. In one embodiment, a possibly changed item may be assigned a new expiration time independent of the evaluation backlog. In another embodiment, if no change is detected, then the item may be assigned a new expiration time as a function of a previous expiration time and on the evaluation backlog.

Claims

1. A network device, comprising:

a transceiver for receiving a plurality of items from one or more client devices over a network; and

a processor for enabling actions, the actions comprising;

for each item in the plurality of items, performing further actions, including;

assigning an expiration time that indicates when an item is to be scheduled for evaluation to detect a possible change to content of the item;

when the assigned expiration time is detected to have expired for the item, scheduling the item for evaluation;

selecting the item that is scheduled for evaluation;

determining a wait time for the selected item that indicates at least a time duration between being scheduled for evaluation and being selected for evaluation;

when a possible change to the content of the selected item is determined for the selected item, setting a new expiration time for scheduling for re-evaluating the selected item for another possible change, wherein the new expiration time being set to at least a minimum expiration time; and

when it is determined that the selected item fails to include a possible change to the content of the selected item, setting the new expiration time for scheduling for re-evaluating the selected item for another possible change that is based in part on the expiration time of the selected item and a function of the wait time for the selected item, wherein each item having the determined possible change is scheduled for re-evaluation at a higher frequency than each item where the possible change is undetermined.

2. The network device of claim 1, wherein when it is determined that the item fails to include a possible change, setting the new expiration time to be a value that is larger than a value of another expiration time of another item that is determined to have a possible change, such that the item is scheduled for re-evaluation at a frequency that is less than a frequency of re-evaluation for the other item.

3. The network device of claim 1, wherein when it is determined that the item fails to include a possible change, setting the new expiration time, further comprises:

when the wait time is determined to be above a threshold value that is based on the expiration time, setting the new expiration time to a value that is based on a multiplicative increase of the expiration time; and

otherwise, setting the new expiration time to be a value that is a decrementing value from the expiration time.

4. The network device of claim 1, wherein when it is determined that the item fails to include a possible change, setting the new expiration time, further comprises:

when the wait time is determined to be above a threshold value that is based on the expiration time, setting the new expiration time to a value that is based on a multiplicative increase of the wait time; and

otherwise, setting the new expiration time to be a value that is a decrementing value from the expiration time.

5. The network device of claim 1, wherein the wait time is configured to represent a time duration consumed by a backlog of other items to be evaluated prior to the item being selected for evaluation.

6. The network device of claim 1, wherein when it is determined that the item fails to include a possible change, setting the new expiration time, further comprises:

employing a plurality of threshold values based on the expiration time, such that each of the plurality of threshold values indicates a different value change of the expiration time; and

when the wait time is determined to be associated with one of the plurality of threshold values, setting the new expiration time to a value that is based on the associated threshold value.

7. The network device of claim 1, wherein determining the wait time for the selected item further indicates a time duration between being assigned the expiration time and being selected for evaluation to detect a possible change.

8. A system, comprising:

one or more network devices that are configured to provide a tool to one or more client devices, such that the tool enables the one or more client devices to provide a plurality of items; and

one or more other network devices that include;

a transceiver configured to receive the plurality of items over a network; and

a processor for enabling actions, the actions comprising;

for each item in the plurality of items, performing further actions, including;

assigning an expiration time that indicates when an item is to be scheduled for evaluation to detect a possible change to content of the item;

when the assigned expiration time is detected to have expired for the item, scheduling the item for evaluation;

selecting the item that is scheduled for evaluation;

determining a wait time for the selected item that indicates at least a time duration between being scheduled for evaluation and being selected for evaluation;

when a possible change to the content of the selected item is determined for the selected item, setting a new expiration time for scheduling for re-evaluating the selected item for another possible change, wherein the new expiration time being set to at least a minimum expiration time; and

when it is determined that the selected item fails to include a possible change to the content of the selected item, setting the new expiration time for scheduling for re-evaluating the selected item for another possible change that is based in part on the expiration time of the selected item and a function of the wait time for the selected item, wherein each item having the determined possible change is scheduled for re-evaluation at a higher frequency than each item where the possible change is undetermined.

9. The system of claim 8, wherein when it is determined that the item fails to include a possible change, setting the new expiration time to be a value that is larger than a value of another expiration time of another item that is determined to have a possible change, such that the item is scheduled for re-evaluation at a frequency that is less than a frequency of re-evaluation for the other item.

10. The system of claim 8, wherein when it is determined that the item fails to include a possible change, setting the new expiration time, further comprises:

when the wait time is determined to be above a threshold value that is based on the expiration time, setting the new expiration time to a value that is based on a multiplicative increase of the expiration time; and

otherwise, setting the new expiration time to be a value that is a decrementing value from the expiration time.

11. The system of claim 8, wherein when it is determined that the item fails to include a possible change, setting the new expiration time, further comprises:

when the wait time is determined to be above a threshold value that is based on the expiration time, setting the new expiration time to a value that is based on a multiplicative increase of the wait time; and

otherwise, setting the new expiration time to be a value that is a decrementing value from the expiration time.

12. The system of claim 8, wherein the wait time is configured to represent a time duration consumed by a backlog of other items to be evaluated prior to the item being selected for evaluation.

13. The system of claim 8, wherein when it is determined that the item fails to include a possible change, setting the new expiration time, further comprises:

employing a plurality of threshold values based on the expiration time, such that each of the plurality of threshold values indicates a different value change of the expiration time; and

when the wait time is determined to be associated with one of the plurality of threshold values, setting the new expiration time to a value that is based on the associated threshold value.

14. The system of claim 13, wherein determining the wait time for the selected item further indicates a time duration between being assigned the expiration time and being selected for evaluation to detect a possible change.