EXTERNAL DATA ACCESS WITH SPLIT INDEX

• US 20150220583A1
• Filed: 01/31/2014
• Published: 08/06/2015
• Est. Priority Date: 01/31/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Teletia, Nikhil, Halverson, Alan D., Patel, Jignesh M., Gankidi, Vinitha Reddy
• Granted Patent: US 9,715,515 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2272 Management thereof
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A split-index can be employed for access to external data. The index can be created on a primary data storage system for data stored externally on a secondary data storage system. After creation, the index can be utilized to expedite at least query execution over the externally stored data. The index can be updated upon detection of changes to data. Further, even when the index is not completely up to date, the index can be exploited for query execution. Furthermore, hybrid execution is enabled with the index and without the index.

Claims

1. A method, comprising:

employing at least one processor configured to execute computer-executable instructions stored in memory to perform the following acts;

creating an index locally in a primary data storage system for data stored externally in a secondary data storage system.

2. The method of claim 1 further comprises acquiring data from the secondary data storage system for at least a portion of a query based on the index.

3. The method of claim 2 further comprises bypassing a native data processing mechanism of the secondary data storage system in acquiring data from the secondary data storage system.

4. The method of claim 1 further comprises restricting query execution with the index based on selectivity of a query.

5. The method of claim 1 further comprises updating the index to reflect changes to the data stored externally in the secondary data storage system.

6. The method of claim 1 further comprises determining one or more updates to the index based on one or more changes to the data stored externally in the secondary data storage system concurrently with query execution.

7. The method of claim 6 further comprising committing the updates to the index upon a temporary pause in processing activity.

8. The method of claim 1 further comprising initiating execution of at least a portion of a query over a first portion of the data with the index and over a second portion of the data without the index.

9. A system, comprising:

a processor coupled to a memory, the processor configured to execute the following computer-executable component stored in the memory;

a first component configured to create an index locally in a primary data storage system for data stored externally in a secondary data storage system.

10. The system of claim 9, the primary data storage system is a relational data warehouse system.

11. The system of claim 10, the secondary data storage system is a non-relational distributed file system.

12. The system of claim 9 further comprises a second component configured to initiate execution of a query over the data stored externally based on the index.

13. The system of claim 12, the second component is further configured to initiate execution of the query over the data stored externally without the index for data that changed since the index was created.

14. The system of claim 9 further comprises a second component configured to initiate execution of a query of data over the data externally stored based on the index and a signal that indicates that the index is to be utilized.

15. The system of claim 9 further comprises a second component configured to update the index in view of changes to the data.

16. The system of claim 15, the second component is configured to perform an incremental update.

17. A computer-readable storage medium having instructions stored thereon that enable at least one processor to perform a method upon execution of the instructions, the method comprising:

creating an index locally in a relational data warehouse system for data stored externally in a non-relational distributed file system.

18. The computer-readable storage medium of claim 17, the method further comprises acquiring at least a first portion of the data from the distributed file system for a query based on the index.

19. The computer-readable storage medium of claim 18, the method further comprises acquiring a second portion of the data from the distributed file system for the query without the index.

20. The computer-readable storage medium of claim 17, the method further comprises incrementally updating the index in response to one or more changes to the data.