SYSTEM AND METHOD FOR FAST FILE TRACKING AND CHANGE MONITORING

• US 20120254128A1
• Filed: 03/30/2011
• Published: 10/04/2012
• Est. Priority Date: 03/30/2011
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Bath, Amritpal Singh, Blank, Mitchell Neuman Jr., Patel, Vishal, Sorkin, Stephen Phillip
• Granted Patent: US 8,548,961 B2
• Time in Patent Office: Days
• US Class Current: 707/689
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases

Abstract

Embodiments are directed towards a dynamic change evaluation mechanism, whereby items having a detected possible change are scheduled for re-evaluation for possible changes at a higher frequency than items detected to not have previously changed, while those items detected as not to have changed are dynamically scheduled for re-evaluation based on an evaluation backlog that may be in turn based, in part, on a time from when an item is assigned an expiration time to when the item is evaluated. In one embodiment, a possibly changed item may be assigned a new expiration time independent of the evaluation backlog. In another embodiment, if no change is detected, then the item may be assigned a new expiration time as a function of a previous expiration time and on the evaluation backlog.

Claims

1. A network device, comprising:

a transceiver for receiving a plurality of items from one or more client devices over a network; and

a processor for enabling actions, the actions comprising;

for each item in the plurality of items, assigning an expiration time that indicates when an item is to be scheduled for evaluation to detect a possible change to the item;

when an expiration time is detected to have expired for an item, scheduling the item for evaluation; and

selecting an item that is scheduled for evaluation, and for the selected item;

determining a wait time for the selected item that indicates at least a time duration between being scheduled for evaluation and being selected for evaluation,

when a possible change is determined for the selected item, setting a new expiration time for scheduling for re-evaluating the selected item for another possible change, the new expiration time being set to at least a minimum expiration time, and

when it is determined that the selected item does not include a possible change, setting the new expiration time for scheduling for re-evaluating the selected item for another possible change that is based in part on the expiration time of the selected item and a function of the wait time for the selected item.

2. The network device of claim 1, wherein when it is determined that the item does not include a possible change, setting the new expiration time to be a value that is larger than a value of an expiration time of an other item that is determined to have a possible change, such that the item is scheduled for re-evaluation at a frequency that is less than a frequency of re-evaluation for the other item.

3. The network device of claim 1, wherein when it is determined that the item does not include a possible change, setting the new expiration time, further comprises:

when the wait time is determined to be above a threshold value that is based on the expiration time, setting the new expiration time to a value that is based on a multiplicative increase of the expiration time; and

otherwise, setting the new expiration time to be a value that is a decrementing value from the expiration time.

4. The network device of claim 1, wherein when it is determined that the item does not include a possible change, setting the new expiration time, further comprises:

when the wait time is determined to be above a threshold value that is based on the expiration time, setting the new expiration time to a value that is based on a multiplicative increase of the wait time; and

otherwise, setting the new expiration time to be a value that is a decrementing value from the expiration time.

5. The network device of claim 1, wherein the wait time is configured to represent a time duration consumed by a backlog of other items to be evaluated prior to the item being selected for evaluation.

6. The network device of claim 1, wherein when it is determined that the item does not include a possible change, setting the new expiration time, further comprises:

employing a plurality of threshold values based on the expiration time, such that each of the plurality of threshold values indicates a different value change of the expiration time; and

when the wait time is determined to be associated with one of the plurality of threshold values, setting the new expiration time to a value that is based on the associated threshold value.

7. The network device of claim 1, wherein-determining the wait time for the selected item further indicates a time duration between being assigned an expiration time and being selected for evaluation to detect a possible change.

8. A method operating on one or more network devices, the method implemented as one or more computer executable instructions for performing actions by the one or more network devices, including:

receiving a plurality of items from one or more client devices;

for each item in the plurality of items, assigning an expiration time that determines when to schedule for evaluation the respective item to detect a change to the item;

when an expiration time is detected to have expired for an item, scheduling the item for evaluation; and

for each item evaluated, setting a new expiration time for the evaluated item to be re-scheduled for re-evaluation, such that an item having a detected change to the item is scheduled for re-evaluation at a higher frequency over an other item for which a change is undetected for the other item, and wherein the new expiration time of the other item is based in part on a backlog of items scheduled for evaluation prior to evaluation of the other item.

9. The method of claim 8, wherein the new expiration time of the other item for which a change is undetected is based in part on a backlog of items scheduled for evaluation prior to evaluation of the other item, further comprises:

determining a wait time for the other item that indicates at least a time duration between being scheduled for evaluation and being evaluated; and

setting the new expiration time of the other item based in part on the expiration time of the other item and a function of the wait time for the other item.

10. The method of claim 9, wherein setting the new expiration time of the other item, further comprises:

when the wait time is determined to be above a threshold value that is based on the expiration time, setting the new expiration time to a value that is based on a multiplicative increase of the expiration time; and

otherwise, setting the new expiration time to be a value that is a decrementing value from the expiration time.

11. The method of claim 9, wherein setting the new expiration time of the other item, further comprises:

when the wait time is determined to be above a threshold value that is based on the expiration time, setting the new expiration time to a value that is based on a multiplicative increase of the wait time; and

otherwise, setting the new expiration time to be a value that is a decrementing value from the expiration time.

12. A system, comprising:

one or more network devices that are configured to provide a tool to one or more client devices, such that the tool enables the one or more client devices to provide a plurality of items; and

one or more other network devices that are configured to receive the plurality of items and to perform actions, the actions comprising;

for each item in the plurality of items, assigning an expiration time that indicates when an item is to be scheduled for evaluation to detect a possible change to the item;

when an expiration time is detected to have expired for an item, scheduling the item for evaluation; and

selecting an item that is scheduled for evaluation, and for the selected item;

determining a wait time for the selected item that indicates at least a time duration between being scheduled for evaluation and being selected for evaluation,

when a possible change is determined for the selected item, setting a new expiration time for scheduling for re-evaluating the selected item for another possible change, the new expiration time being set to at least a minimum expiration time, and

when it is determined that the selected item does not include a possible change, setting the new expiration time for scheduling for re-evaluating the selected item for another possible change that is based in part on the expiration time of the selected item and a function of the wait time for the selected item.

13. The system of claim 12, wherein when it is determined that the item does not include a possible change, setting the new expiration time to be a value that is larger than a value of an expiration time of an other item that is determined to have a possible change, such that the item is scheduled for re-evaluation at a frequency that is less than a frequency of re-evaluation for the other item.

14. The system of claim 12, wherein when it is determined that the item does not include a possible change, setting the new expiration time, further comprises:

when the wait time is determined to be above a threshold value that is based on the expiration time, setting the new expiration time to a value that is based on a multiplicative increase of the expiration time; and

otherwise, setting the new expiration time to be a value that is a decrementing value from the expiration time.

15. The system of claim 12, wherein when it is determined that the item does not include a possible change, setting the new expiration time, further comprises:

when the wait time is determined to be above a threshold value that is based on the expiration time, setting the new expiration time to a value that is based on a multiplicative increase of the wait time; and

otherwise, setting the new expiration time to be a value that is a decrementing value from the expiration time.

16. The system of claim 12, wherein the wait time is configured to represent a time duration consumed by a backlog of other items to be evaluated prior to the item being selected for evaluation.

17. The system of claim 12, wherein when it is determined that the item does not include a possible change, setting the new expiration time, further comprises:

employing a plurality of threshold values based on the expiration time, such that each of the plurality of threshold values indicates a different value change of the expiration time; and

when the wait time is determined to be associated with one of the plurality of threshold values, setting the new expiration time to a value that is based on the associated threshold value.

18. The system of claim 17, wherein determining the wait time for the selected item further indicates a time duration between being assigned an expiration time and being selected for evaluation to detect a possible change.

19. A non-transitory, computer-readable storage device that includes data and instructions, that when installed within on a computer device causes the computer device to perform actions, comprising:

receiving a plurality of items from one or more client devices;

for each item in the plurality of items, assigning an expiration time that determines when to schedule for evaluation the respective item to detect a change to the item;

when an expiration time is detected to have expired for an item, scheduling the item for evaluation; and

for each item evaluated, setting a new expiration time for the evaluated item to be re-scheduled for re- evaluation, such that an item having a detected change to the item is scheduled for re-evaluation at a higher frequency over an other item for which a change is undetected for the other item, and wherein the new expiration time of the other item is based in part on a backlog of items scheduled for evaluation prior to evaluation of the other item.

20. The non-transitory, computer-readable storage device of claim 19, wherein the new expiration time of the other item for which a change is undetected is based in part on a backlog of items scheduled for evaluation prior to evaluation of the other item, further comprises:

determining a wait time for the other item that indicates at least a time duration between being scheduled for evaluation and being evaluated; and

setting the new expiration time of the other item based in part on the expiration time of the other item and a function of the wait time for the other item.

21. The non-transitory, computer-readable storage device of claim 20, wherein setting the new expiration time of the other item, further comprises:

when the wait time is determined to be above a threshold value that is based on the expiration time, setting the new expiration time to a value that is based on a multiplicative increase of the expiration time; and

otherwise, setting the new expiration time to be a value that is a decrementing value from the expiration time.

22. The non-transitory, computer-readable storage device of claim 20, wherein setting the new expiration time of the other item, further comprises:

when the wait time is determined to be above a threshold value that is based on the expiration time, setting the new expiration time to a value that is based on a multiplicative increase of the wait time; and

otherwise, setting the new expiration time to be a value that is a decrementing value from the expiration time.