CACHING EXTERNAL DATA SOURCES FOR SQL PROCESSING

• US 20140317084A1
• Filed: 04/19/2013
• Published: 10/23/2014
• Est. Priority Date: 04/19/2013
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Iyer, Chandrasekharan, Chaudhry, Atif
• Granted Patent: US 9,251,210 B2
• Time in Patent Office: Days
• US Class Current: 707/713
• CPC Class Codes: G06F 16/24539 using cached or materialised query results
• CPC Class Codes: G06F 16/24552 Database cache management

Abstract

Techniques are provided for caching external data sources for SQL processing. According to an embodiment, a database system receives a query that requires foreign data from an external data source. In response to receiving the query, the database system determines whether the foreign data currently resides in a cache of the database system. In response to determining that the foreign does not currently reside in the cache, the database system retrieves the foreign data from the external data source, maps the foreign data to a column granule of a table, and stores the column granule in the cache. In another embodiment, the database system receives a second query that requires the foreign data. In response to determining that the foreign data is cached with the stored column granule, the database system retrieves the column granule from the cache.

Claims

1. A method for optimizing query processing, the method comprising:

receiving, at a database system, a query that requires foreign data from an external data source;

in response to receiving the query, determining whether the foreign data currently resides in a cache of the database system;

in response to determining that the foreign data does not currently reside in the cache;

retrieving the foreign data from the external data source;

mapping the foreign data to a column granule of a table; and

storing the column granule in the cache;

wherein the method is performed by one or more computing devices.

2. The method of claim 1,

wherein mapping the foreign data to the column granule is based on metadata, stored in a database of the database system, defining a layout of the table and how foreign data maps to the layout of the table;

wherein the database does not store the foreign data.

3. The method of claim 1, wherein the query is a first query, the method further comprising:

receiving, at the database system, a second query that requires second foreign data from the external source;

in response to receiving the second query, determining whether the second foreign data currently resides in the cache of the database system;

in response to determining that the second foreign data does not currently reside in the cache;

retrieving the foreign data from the external data source;

mapping the foreign data to a column granule of a table; and

storing the column granule in the cache.

4. The method of claim 1, wherein the column granule is an entire column referenced by the query.

5. The method of claim 1,

wherein the column granule is a first portion of a column referenced by the query, wherein the cache is a first result cache of a first database server;

wherein a second result cache of a second database server stores a second column granule;

wherein the second column granule is a second portion of the column referenced by the query.

6. The method of claim 1, wherein the query is a first query, the method further comprising:

receiving, at the database system, a second query that requires the foreign data from the external source;

wherein the second query includes a different expression than the first query;

in response to receiving the second query, determining whether the foreign data currently resides in the cache of the database system;

in response to determining that the foreign data currently resides in the cache;

retrieving, from the cache, a column granule populated with the foreign data currently residing in the cache.

7. The method of claim 1, wherein the query is a first query, the method further comprising:

receiving, at the database system, a second query that requires the foreign data from the external source;

in response to receiving the second query, determining whether the foreign data currently resides in the cache of the database system;

querying the external data source to determine whether the foreign data is still valid;

in response to determining that the foreign data is no longer valid;

retrieving updated foreign data from the external data source;

mapping the updated foreign data to the column granule of the table; and

storing the column granule with the updated foreign data in the cache.

8. The method of claim 1, further comprising:

receiving, at the database system, a notification from the external data source that the foreign data in the external source has changed;

in response to receiving the notification, invalidating the column granule in the data cache.

9. The method of claim 1, further comprising:

determining statistics associated with the foreign data;

storing, in the cache, the statistics associated with the foreign data.

10. The method of claim 1, further comprising:

evicting a second column granule of the table from the cache;

wherein evicting the second column granule of the table comprises;

deleting the second column granule from the cache;

generating a temporary file to store the second column granule;

storing a pointer to the temporary file in the cache.

11. The method of claim 1,

wherein the first column granule is part of a view of the table that has been annotated to enable caching of foreign data;

wherein a second column granule which does not belong to the view of the table that has been annotated is not stored in the cache.

12. One or more non-transitory computer-readable media storing instructions, which, in response to being executed by one or more processors, cause one or more computing devices to perform operations comprising:

receiving, at a database system, a query that requires foreign data from an external data source;

in response to receiving the query, determining whether the foreign data currently resides in a cache of the database system;

in response to determining that the foreign data does not currently reside in the cache;

retrieving the foreign data from the external data source;

mapping the foreign data to a column granule of a table; and

storing the column granule in the cache.

13. The non-transitory computer-readable media of claim 12,

wherein mapping the foreign data to the column granule is based on metadata, stored in a database of the database system, defining a layout of the table and how foreign data maps to the layout of the table;

wherein the database does not store the foreign data.

14. The non-transitory computer-readable media of claim 12, wherein the query is a first query, the operations further comprising:

receiving, at the database system, a second query that requires second foreign data from the external source;

in response to receiving the second query, determining whether the second foreign data currently resides in the cache of the database system;

in response to determining that the second foreign data does not currently reside in the cache;

retrieving the foreign data from the external data source;

mapping the foreign data to a column granule of a table; and

storing the column granule in the cache.

15. The non-transitory computer-readable media of claim 12, wherein the column granule is an entire column referenced by the query.

16. The non-transitory computer-readable media of claim 12,

wherein the column granule is a first portion of a column referenced by the query;

wherein the cache is a first result cache of a first database server;

wherein a second result cache of a second database server stores a second column granule;

wherein the second column granule is a second portion of the column referenced by the query.

17. The non-transitory computer-readable media of claim 12, wherein the query is a first query, the operations further comprising:

receiving, at the database system, a second query that requires the foreign data from the external source;

wherein the second query includes a different expression than the first query;

in response to receiving the second query, determining whether the foreign data currently resides in the cache of the database system;

in response to determining that the foreign data currently resides in the cache;

retrieving, from the cache, the column granule currently residing in the cache.

18. The non-transitory computer-readable media of claim 12, wherein the query is a first query, the operations further comprising:

receiving, at the database system, a second query that requires the foreign data from the external source;

in response to receiving the second query, determining whether the foreign data currently resides in the cache of the database system;

querying the external data source to determine whether the foreign data is still valid;

in response to determining that the foreign data is no longer valid;

retrieving updated foreign data from the external data source;

mapping the updated foreign data to the column granule of the table; and

storing the column granule populated with the updated foreign data in the cache.

19. The non-transitory computer-readable media of claim 12, further comprising:

receiving, at the database system, a notification from the external data source that the foreign data in the external source has changed;

in response to receiving the notification, invalidating the column granule in the cache.

20. The non-transitory computer-readable media of claim 12, further comprising:

determining statistics associated with the foreign data;

storing, in the cache, the statistics associated with the foreign data.

21. The non-transitory computer-readable media of claim 12, wherein the instructions further cause the one or more computing devices to perform operations comprising:

evicting a second column granule of the table from the cache;

wherein evicting the second column granule of the table comprises;

deleting the second column granule from the cache;

generating a temporary file to store the second column granule;

storing a pointer to the temporary file in the cache.

22. The non-transitory computer-readable media of claim 12,

wherein the first column granule is part of a view of the table that has been annotated to enable caching of foreign data;

wherein a second column granule for which does not belong to the view of the table that has been annotated is not stored in the cache.