Checkpointing Iterators During Search

• US 20090070308A1
• Filed: 08/29/2008
• Published: 03/12/2009
• Est. Priority Date: 08/31/2007
• Status: Active Grant

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Powerset (Microsoft Corporation)
• Inventors: Joffe, Nitay, Biewald, Lukas, James, Andrew Alan, Walters, Chad
• Granted Patent: US 8,316,036 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Tools and techniques are described herein for checkpointing iterators during search. These tools may provide methods that include instantiating iterators in response to a search request. The iterators include fixed state information that remains constant over a life of the iterator, and further include dynamic state information that is updated over the life of the iterator. The iterators traverse through postings lists in connection with performing the search request. As the iterators traverse the posting lists, the iterators may update their dynamic state information. The iterators may then evaluate whether to create checkpoints, with the checkpoints including representations of the dynamic state information.

Claims

1. At least one computer-readable storage medium having computer-executable instructions stored thereon which, when executed by a computer system, cause the computer system to perform a method comprising:

instantiating at least one iterator in response to a search request, wherein the iterator includes fixed state information that remains constant over a life of the iterator, and includes dynamic state information that is updated over the life of the iterator;

traversing the iterator through at least a portion of at least one postings list in connection with performing the search request;

updating at least one instance of the dynamic state information, in response to traversing the iterator through at least the portion of the postings list; and

evaluating whether to create a checkpoint of the iterator, wherein the checkpoint includes at least a representation of the dynamic state information.

2. The storage medium of claim 1, further comprising instructions for storing the representation of only the dynamic state information, as associated with the checkpoint.

3. The storage medium of claim 1, further comprising instructions for storing data from which the dynamic state information may be calculated when restoring the checkpoint of the iterator.

4. The storage medium of claim 1, further comprising instructions for evaluating whether the iterator is associated with at least one child iterator.

5. The storage medium of claim 1, further comprising instructions for sending a command to at least one child iterator, associated with the iterator, to create a further checkpoint associated with the child iterator, wherein the further checkpoint includes at least a further representation of further dynamic state information maintained by the child iterator.

6. The storage medium of claim 1, wherein the instructions for evaluating whether to create a checkpoint include instructions for directing the iterator to create a checkpoint, in response to a checkpoint command received from a parent iterator.

7. The storage medium of claim 1, wherein the instructions for evaluating whether to create a checkpoint include instructions for directing the iterator to create a checkpoint.

8. The storage medium of claim 1, further comprising instructions for creating a storage structure associated with the iterator, and further comprising storing the representation of the dynamic state information in the storage structure.

9. The storage medium of claim 8, wherein the instructions for creating a storage structure include instructions for creating a stack structure that includes a plurality of entries, further comprising storing the representation of the dynamic state information in one of the entries, and further comprising storing at least a further representation of a further instance of the dynamic state information in a further one of the entries.

10. At least one computer-readable storage medium having computer-executable instructions stored thereon which, when executed by a computer system, cause the computer system to perform a method comprising:

traversing an iterator through at least a portion of at least one postings list in response to at least one search request, wherein the iterator includes fixed state information that remains constant over a life of the iterator, and dynamic state information that is updated over the life of the iterator, wherein the iterator includes a storage structure for including a representation of at least one checkpoint, wherein the checkpoint represents a least one instance of the dynamic state information;

updating at least one instance of the dynamic state information in response to traversing the iterator through at least a portion of the postings list; and

evaluating whether to restore if the iterator to a previous state represented by the dynamic state information stored with the checkpoint.

11. The storage medium of claim 10, further comprising instructions for loading the dynamic state information from the storage structure into the iterator, so as to restore the iterator to the previous state.

12. The storage medium of claim 10, further comprising instructions for calculating the dynamic state information based on information stored in the storage structure when checkpointing the iterator.

13. The storage medium of claim 10, further comprising instructions for evaluating whether the iterator is associated with at least one child iterator.

14. The storage medium of claim 13, further comprising instructions for sending a command to the at least one child iterator, directing the child iterator to restore dynamic state information maintained by the child iterator.

15. The storage medium of claim 10, wherein the instructions for evaluating whether to restore the iterator include instructions for directing the iterator to restore dynamic state information from at least one checkpoint, in response to a restore command received from a parent iterator.

16. The storage medium of claim 10, wherein the instructions for evaluating whether to restore the iterator include instructions for directing the iterator to restore dynamic state information from at least one previously-stored checkpoint.

17. At least one computer-readable storage medium having computer-executable instructions stored thereon which, when executed by a computer system, cause the computer system to perform a method comprising:

creating a plurality of iterators, wherein one of the iterators is a parent iterator operating at least a further one of the iterators as a child iterator, wherein the parent iterator includes;

first fixed state information that remains constant over a life of the parent iterator,

first dynamic state information that is updated over the life of the parent iterator, and

a first checkpoint mechanism that includes a first stack structure associated with a plurality of entries, wherein the first checkpoint mechanism is responsive to an internal checkpoint command to copy at least one instance of only the first dynamic state information, and not the first fixed state information, into one of the entries of the first stack structure, and to update a first counter mechanism to indicate that a first checkpoint including only the first dynamic state information has been copied into the first stack structure;

wherein the child iterator includes;

second fixed state information that remains constant over a life of the child iterator,

second dynamic state information that is updated over the life of the child iterator, and

a second checkpoint mechanism that includes a second stack structure associated with a plurality of entries, wherein the second checkpoint mechanism is responsive to an external checkpoint command received from the parent iterator to copy at least one instance of only the second dynamic state information, and not the second fixed state information, into one of the entries of the second stack structure, and to update a second counter mechanism to indicate that a second checkpoint including the second dynamic state information has been copied into the second stack structure;

traversing at least the child iterator through at least a portion of at least one postings list in response to at least one search request;

updating at least one instance of the second dynamic state information in response to traversing the child iterator through at least a portion of the postings list;

storing the first checkpoint in the first stack structure, wherein the first checkpoint includes the first dynamic state information;

storing the second checkpoint in the second stack structure, wherein the second checkpoint includes the second dynamic state information;

restoring at least the child iterator to a previous state by copying the second dynamic state information from the second checkpoint; and

restoring at least the parent iterator to the previous state by copying the first dynamic state information from the first checkpoint.

18. The storage medium of claim 17, further comprising instructions for creating at least a further child iterator, wherein the child iterator includes:

third fixed state information that remains constant over a life of the further child iterator,

third dynamic state information that is updated over the life of the further child iterator, and

a third checkpoint mechanism that includes a third stack structure having a plurality of entries, wherein the third checkpoint mechanism is responsive to an external checkpoint command received from the parent iterator to copy at least one instance of only the third dynamic state information, and not the third fixed state information, into one of the entries of the third stack structure, and to update a third counter mechanism to indicate that a third checkpoint including the third dynamic state information has been copied into the third stack structure.

19. The storage medium of claim 18, further comprising instructions for:

traversing at least the further child iterator through at least a portion of at least a further postings list in response to the search request;

updating at least one instance of the third dynamic state information in response to traversing the further child iterator through at least a portion of the further postings list;

storing the third checkpoint in the third stack structure, wherein the third checkpoint includes the third dynamic state information; and

restoring at least the further child iterator to the previous state by copying the third dynamic state information from the third checkpoint.

20. The storage medium of claim 19, wherein the parent iterator is for:

receiving first document information from the child iterator, wherein the first document information is extracted from the postings list,

receiving further document information from the further child iterator, wherein the further document information is extracted from the further postings list, and

performing at least one combinatorial operation on the first document information and the further document information to provide results responsive to the search request.