System and methods for invalidation to enable caching of dynamically generated content

• US 6,889,234 B1
• Filed: 02/26/2002
• Issued: 05/03/2005
• Est. Priority Date: 02/26/2001
• Status: Expired due to Fees

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Li, Wen-Syan, Candan, Kasim Selcuk, Agrawal, Divyakant
• Primary Examiner(s): ALAM, SHAHID AL
• Application Number:
  US10/084,630
• Time in Patent Office: 1,162 Days
• Field of Search: 707/2, 707/3, 707/9, 707/10, 707/104.1, 707/203, 707/100, 709/203, 709/217, 709/219, 709/226
• US Class Current: 1/1
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: Y10S 707/959 Network
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99954 Version management

Abstract

Various systems and methods are describe for determining dynamically generated content to be invalidated as a result of changes to underlying data. In one method, invalidation processing is performed using a delayed version of the database that reflects the state of the database at the beginning of a preceding processing cycle, and an update log that reflects all updates since the beginning of the preceding processing cycle. In another method, invalidation processing is performed by locking a database which is locked during processing so as to reflect the state of the database at the beginning of processing, and an update log that reflects all updates to the database since the beginning of the preceding processing cycle. In another method, invalidation processing uses the database while it is freely updated during said processing, and the update log of the database.

Claims

1. In a system for producing dynamically generated content in response to user content requests that generate database queries, a method for invalidating dynamically generated content stored in caches of network devices, comprising:

receiving updates to a database of the system;

periodically performing processing on a set of received queries to identify queries for which corresponding dynamically generated content will be invalidated as a result of said updates; and

sending invalidation messages to devices storing said corresponding content of said identified queries,

wherein said processing is performed using a delayed version of said database that reflects the state of the database at the beginning of a preceding processing cycle, and an update log that reflects all updates since the beginning of the preceding processing cycle.

2. The method claimed in claim 1, wherein said processing comprises:

determining whether respective results sets for subsets of said set of queries are empty or non-empty for tuples added to and deleted from said database between the beginning of a preceding processing cycle and the beginning of the current processing cycle; and

for each non-empty results set, designating the queries of the corresponding subset as ones of said identified queries.

3. The method claimed in claim 2, further comprising, if a results set for a given query Q is non-empty, designating as identified queries one or more additional queries that are dependent from said query Q in a positive query lattice.

4. The method claimed in claim 2, further comprising, if a results set for a given query Q is non-empty, skipping processing of one or more additional queries that are dependent from said query Q in a negative query lattice.

5. The method claimed in claim 1, wherein a subset of said queries comprises queries represented by a common query type, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

6. The method claimed in claim 1, wherein a subset of said queries comprises queries represented by a common query type, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type, and to update tables constructed from said update log and corresponding to relations utilized by queries of said query type, for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

7. The method claimed in claim 1, wherein a subset of said queries comprises queries represented by a common query type and including a join operation, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type, and to update tables constructed from said update log and corresponding to relations utilized by queries of said query type, and to relations of said database utilized by queries of said query type for which there have been no updates since the preceding invalidation cycle, for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

8. A system for producing dynamically generated content in response to user content requests that generate database queries, comprising:

a database management system;

a web server for receiving user requests for dynamically generated content and for providing said dynamically generated content; and

an application server coupled between the database management system and the web server for generating database polling queries corresponding to said requests and for supplying resulting data from the database to said web server,

the system performing an invalidation method to invalidate dynamically generated content generated by said system and stored in caches of network devices, said method comprising;

receiving updates to the database management system;

periodically performing processing on a set of received queries to identify queries for which corresponding dynamically generated content will be invalidated as a result of said updates; and

sending invalidation messages to devices storing said corresponding content of said identified queries,

wherein said processing is performed using a delayed version of said database that reflects the state of the database at the beginning of a preceding processing cycle, and an update log that reflects all updates since the beginning of the preceding processing cycle.

9. The system claimed in claim 8, wherein said processing comprises:

determining whether respective results sets for subsets of said set of queries are empty or non-empty for tuples added to and deleted from said database between the beginning of a preceding processing cycle and the beginning of the current processing cycle; and

for each non-empty results set, designating the queries of the corresponding subset as ones of said identified queries.

10. The system claimed in claim 8, said processing further comprising, if a results set for a given query Q is non-empty, designating as identified queries one or more additional queries that are dependent from said query Q in a positive query lattice.

11. The system claimed in claim 8, said processing further processing of one or more additional queries that are dependent from said query Q in a negative query lattice.

12. The system claimed in claim 8, wherein a subset of said queries comprises queries represented by a common query type, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

13. The system claimed in claim 8, wherein a subset of said queries comprises queries represented by a common query type, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type, and to update tables constructed from said update log and corresponding to relations utilized by queries of said query type, for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

14. The system claimed in claim 8, wherein a subset of said queries comprises queries represented by a common query type and including a join operation, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type, and to update tables constructed from said update log and corresponding to relations utilized by queries of said query type, and to relations of said database utilized by queries of said query type for which there have been no updates since the preceding invalidation cycle, for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

15. In a system for producing dynamically generated content in response to user content requests that generate database queries, a method for invalidating dynamically generated content stored in caches of network devices, comprising:

receiving updates to a database of the system;

periodically performing processing on a set of received queries to identify queries for which corresponding dynamically generated content will be invalidated as a result of said updates; and

sending invalidation messages to devices storing said corresponding content of said identified queries,

wherein said processing is performed using said database, which is prevented from being updated during said processing so as to reflect the state of the database at the beginning of said processing, and an update log that reflects all updates to the database since the beginning of the preceding processing cycle.

16. The method claimed in claim 15, wherein said processing comprises:

determining whether respective results sets for subsets of said set of queries are empty or non-empty for tuples added to and deleted from said database between the beginning of a preceding processing cycle and the beginning of the current processing cycle, using said locked database and said update log of said data; and

for each non-empty results set, designating the queries of the corresponding subset as ones of said identified queries.

17. The method claimed in claim 16, further comprising, if a results set for a given query Q is non-empty, designating as identified queries one or more additional queries that are dependent from said query Q in a positive query lattice.

18. The method claimed in claim 16, further comprising, if a results set for a given query Q is non-empty, skipping processing of one or more additional queries that are dependent from said query Q in a negative query lattice.

19. The method claimed in claim 15, wherein a subset of said queries comprises queries represented by a common query type, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

20. The method claimed in claim 15, wherein a subset of said queries comprises queries represented by a common query type, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type, and to update tables constructed from said update log and corresponding to relations utilized by queries of said query type, for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

21. The method claimed in claim 15, wherein a subset of said queries comprises queries represented by a common query type and including a join operation, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type, and to update tables constructed from said update log and corresponding to relations utilized by queries of said query type, and to relations of said database utilized by queries of said query type for which there have been no updates since the preceding invalidation cycle, for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

22. A system for producing dynamically generated content in response to user content requests that generate database queries, comprising:

a database management system;

a web server for receiving user requests for dynamically generated content and for providing said dynamically generated content; and

an application server coupled between the database management system and the web server for generating database polling queries corresponding to said requests and for supplying resulting data from the database to said web server,

the system performing an invalidation method to invalidate dynamically generated content generated by said system and stored in caches of network devices, said method comprising;

receiving updates to the database management system;

periodically performing processing on a set of received queries to identify queries for which corresponding dynamically generated content will be invalidated as a result of said updates; and

sending invalidation messages to devices storing said corresponding content of said identified queries,

wherein said processing is performed using said database, which is prevented from being updated during said processing so as to reflect the state of the database at the beginning of said processing, and an update log that reflects all updates to the database since the beginning of the preceding processing cycle.

23. The system claimed in claim 22, wherein said processing comprises:

determining whether respective results sets for subsets of said set of queries are empty or non-empty for tuples added to and deleted from said database between the beginning of a preceding processing cycle and the beginning of the current processing cycle, using said locked database and said update log of said database; and

for each non-empty results set, designating the queries of the corresponding subset as ones of said identified queries.

24. The system claimed in claim 23, said processing further comprising, if a results set for a given query Q is non-empty, designating as identified queries one or more additional queries that are dependent from said query Q in a positive query lattice.

25. The system claimed in claim 23, said processing further comprising, if a results set for a given query Q is non-empty, skipping processing of one or more additional queries that are dependent from said query Q in a negative query lattice.

26. The system claimed in claim 22, wherein a subset of said queries comprises queries represented by a common query type, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

27. The system claimed in claim 22, wherein a subset of said queries comprises queries represented by a common query type, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type, and to update tables constructed from said update log and corresponding to relations utilized by queries of said query type, for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

28. The system claimed in claim 22, wherein a subset of said queries comprises queries represented by a common query type and including a join operation, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type, and to update tables constructed from said update log and corresponding to relations utilized by queries of said query type, and to relations of said database utilized by queries of said query type for which there have been no updates since the preceding invalidation cycle, for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

29. In a system for producing dynamically generated content in response to user content requests that generate database queries, a method for invalidating dynamically generated content stored in caches of network devices, comprising:

receiving updates to a database of the system;

periodically performing processing on a set of received queries to identify queries for which corresponding dynamically generated content will be invalidated as a result of said updates; and

sending invalidation messages to devices storing said corresponding content of said identified queries,

wherein said processing is performed using said database, which is freely updated during said processing, and a portion of an update log of said database that reflects updates to the database made prior to the beginning of said processing.

30. The method claimed in claim 29, wherein said processing comprises:

determining whether respective results sets for subsets of said set of queries are empty or non-empty for tuples added to and deleted from said database since the beginning of a preceding processing cycle, using said freely updated database and said locked update log of said database; and

for each non-empty results set, designating the queries of the corresponding subset as ones of said identified queries.

31. The method claimed in claim 30, further comprising, if a results set for a given query Q is non-empty, designating as identified queries one or more additional queries that are dependent from said query Q in a positive query lattice.

32. The method claimed in claim 30, further comprising, if a results set for a given query Q is non-empty, skipping processing of one or more additional queries that are dependent from said query Q in a negative query lattice.

33. The method claimed in claim 29, wherein a subset of said queries comprises queries represented by a common query type, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

34. The method claimed in claim 29, wherein a subset of said queries comprises queries represented by a common query type, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type, and to update tables constructed from said update log and corresponding to relations utilized by queries of said query type, for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

35. The method claimed in claim 29, wherein a subset of said queries comprises queries represented by a common query type and including a join operation, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type, and to update tables constructed from said update log and corresponding to relations utilized by queries of said query type, and to relations of said database utilized by queries of said query type for which there have been no updates since the preceding invalidation cycle, for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

36. The method claimed in claim 29, wherein said processing further comprises correcting an under-invalidation of a preceding processing cycle.

37. A system for producing dynamically generated content in response to user content requests that generate database queries, comprising:

a database management system;

a web server for receiving user requests for dynamically generated content and for providing said dynamically generated content; and

an application server coupled between the database management system and the web server for generating database polling queries corresponding to said requests and for supplying resulting data from the database to said web server,

the system performing an invalidation method to invalidate dynamically generated content generated by said system and stored in caches of network devices, said method comprising;

receiving updates to the database management system;

periodically performing processing on a set of received queries to identify queries for which corresponding dynamically generated content will be invalidated as a result of said updates; and

sending invalidation messages to devices storing said corresponding content of said identified queries,

wherein said processing is performed using said database, which is freely updated during said processing, and a portion of an update log of said database that reflects updates to the database made prior to the beginning of said processing.

38. The system claimed in claim 37, wherein said processing comprises:

determining whether respective results sets for subsets of said set of queries are empty or non-empty for tuples added to and deleted from said database since the beginning of a preceding processing cycle, using said freely updated database and said locked update log of said database; and

for each non-empty results set, designating the queries of the corresponding subset as ones of said identified queries.

39. The system claimed in claim 38, further comprising, if a results set for a given query Q is non-empty, designating as identified queries one or more additional queries that are dependent from said query Q in a positive query lattice.

40. The system claimed in claim 38, further comprising, if a results set for a given query Q is non-empty, skipping processing of one or more additional queries that are dependent from said query Q in a negative query lattice.

41. The system claimed in claim 37, wherein a subset of said queries comprises queries represented by a common query type, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

42. The system claimed in claim 37, wherein a subset of said queries comprises queries represented by a common query type, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type, and to update tables constructed from said update log and corresponding to relations utilized by queries of said query type, for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

43. The system claimed in claim 37, wherein a subset of said queries comprises queries represented by a common query type and including a join operation, and

wherein processing of said subset comprises:

issuing polling queries to a query instance table of said query type, and to update tables constructed from said update log and corresponding to relations utilized by queries of said query type, and to relations of said database utilized by queries of said query type for which there have been no updates since the preceding invalidation cycle, for each of said updates; and

designating query instances identified through said polling queries as said identified queries.

44. The system claimed in claim 37, wherein said processing further comprises correcting an under-invalidation of a preceding processing cycle.