Optimizing data processing using dynamic schemas

• US 9,817,877 B2
• Filed: 11/03/2015
• Issued: 11/14/2017
• Est. Priority Date: 07/11/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Taranov, Igor
• Primary Examiner(s): Ly, Cheyne D
• Application Number:
  US14/931,017
• Publication Number:
  US 20160055223A1
• Time in Patent Office: 742 Days
• Field of Search: 707756, 707807
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/2219 Large Object storage; Management thereof
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 9/30192 according to data descriptor, e.g. dynamic data typing

Abstract

A computer system access feed data belonging to a particular feed, and executes a dynamic server statement to create a relational dataset with data type fields from the feed data in an in-memory table of the server. The dynamic server statement is stored within metadata associated with the feed. The computer system also applies a second dynamic server statement to the data feed which applies one or more data processing conditions indicated in the metadata. The second dynamic server statement is also stored within the metadata associated with the feed. Upon determining that one or more feed data rows match the data processing conditions, the computer system places feed data row information about the matching data rows into an alert table that includes references to a regional blob table with blob data, thereby triggering an alert.

Claims

1. A computer system, comprising:

one or more processors;

system memory; and

one or more computer-readable storage media having stored thereon computer-executable instructions that are executable by the one or more processors to configure the computer system to efficiently process data using dynamic schemas, the computer-executable instructions including instructions that are executable to configure the computer system to perform at least the following;

access feed data belonging to a particular feed comprising a plurality of records;

convert one or more records of the plurality of records into a binary blob data table including, for each record being converted;

creating a record containing at least one record identifier; and

creating a binary blob that includes binary information representing a plurality of fields within the record and including metadata indicating data processing conditions for processing the plurality of fields;

execute a dynamic server statement on the binary blob data table to create a relational dataset with data type fields from the one or more converted records of the binary blob data table in an in-memory table of the server, wherein the dynamic server statement is stored within the metadata associated with the binary blob of the one or more converted records;

apply a second dynamic server statement to the in-memory table to create an alert table based on at least one or more of the data processing conditions indicated in the metadata, wherein the second dynamic server statement is stored within the metadata associated with the binary blob; and

upon determining that a binary blob of the one or more converted records matches the data processing conditions, place a reference to the matching binary blob data table record into an alert table, thereby triggering an alert.

2. The computer system of claim 1, wherein the feed data is processed in the system memory, and not in a corresponding static table.

3. The computer system of claim 1, wherein the alert table includes reference to a database table with a binary feed data rather than static representations of the feed fields.

4. The computer system of claim 1, wherein one or more portions of at least one of the dynamic server statement or the second dynamic server statement is taken from the metadata.

5. The computer system of claim 1, wherein at least one of the dynamic server statement or the second dynamic server is configured to reformat incoming blob data to represent one row of data with one binary value in a blob column of a database table.

6. The computer system of claim 1, wherein at least one of the dynamic server statement or the second dynamic server is configured to read the feed data belonging to a particular feed reading session from a database table.

7. The computer system of claim 1, wherein items placed in the alert table automatically cause an alert to be issued to any user that has subscribed to receive those alerts.

8. A method, implemented at a computer system that includes one or more processors, for efficiently processing data using dynamic schemas, the method comprising:

accessing feed data belonging to a particular feed comprising a plurality of records;

converting one or more records of the plurality of records into a binary blob data table including, for each record being converted;

creating a record containing at least one record identifier; and

creating a binary blob that includes binary information representing a plurality of fields within the record and including metadata indicating data processing conditions for processing the plurality of fields;

executing a dynamic server statement on the binary blog data table to create a relational dataset with data type fields from the one or more converted records of the binary blob data table in an in-memory table of the server, wherein the dynamic server statement is stored within the metadata associated with the binary blob of the one or more converted records;

applying a second dynamic server statement to the in-memory table to create an alert table based on at least one or more of the data processing conditions indicated in the metadata, wherein the second dynamic server statement is stored within the metadata associated with the binary blob; and

upon determining that a binary blob of the one or more converted records matches the data processing conditions, placing a reference to the matching binary blob data table record into an alert table, thereby triggering an alert.

9. The method of claim 8, wherein the feed data is processed in system memory, and not in a corresponding static table.

10. The method of claim 8, wherein the alert table includes reference to a database table with a binary feed data rather than static representations of the feed fields.

11. The method of claim 8, wherein one or more portions of at least one of the dynamic server statement or the second dynamic server statement is taken from the metadata.

12. The method of claim 8, wherein at least one of the dynamic server statement or the second dynamic server is configured to reformat incoming blob data to represent one row of data with one binary value in a blob column of a database table.

13. The method of claim 8, wherein at least one of the dynamic server statement or the second dynamic server is configured to read the feed data belonging to a particular feed reading session from a database table.

14. The method of claim 8, wherein items placed in the alert table automatically cause an alert to be issued to any user that has subscribed to receive those alerts.

15. A computer program product comprising one or more hardware storage devices having stored thereon computer-executable instructions that are executable by one or more processors to configure a computer system to efficiently process data using dynamic schemas, the computer-executable instructions including instructions that are executable to configure the computer system to perform a least the following:

access feed data belonging to a particular feed comprising a plurality of records;

convert one or more records of the plurality of records into a binary blob data table including, for each record being converted;

creating a record containing at least one record identifier; and

creating a binary blob that includes binary information representing a plurality of fields within the record and including metadata indicating data processing conditions for processing the plurality of fields;

execute a dynamic server statement on the binary blob data table to create a relational dataset with data type fields from the one or more converted records of the binary blob data table in an in-memory table of the server, wherein the dynamic server statement is stored within the metadata associated with the binary blob of the one or more converted records;

apply a second dynamic server statement to the in-memory table to create an alert table based on at least one or more of the data processing conditions indicated in the metadata, wherein the second dynamic server statement is stored within the metadata associated with the binary blob; and

upon determining that a binary blob of the one or more converted records matches the data processing conditions, place a reference to the matching binary blob data table record into an alert table, thereby triggering an alert.

16. The computer program product of claim 15, wherein the feed data is processed in system memory, and not in a corresponding static table.

17. The computer program product of claim 15, wherein the alert table includes reference to a database table with a binary feed data rather than static representations of the feed fields.

18. The computer program product of claim 15, wherein one or more portions of at least one of the dynamic server statement or the second dynamic server statement is taken from the metadata.

19. The computer program product of claim 15, wherein at least one of the dynamic server statement or the second dynamic server is configured to perform at least one of:

(i) reformat incoming blob data to represent one row of data with one binary value in a blob column of a database table or (ii) read the feed data belonging to a particular feed reading session from a database table.

20. The computer program product of claim 15, wherein items placed in the alert table automatically cause an alert to be issued to any user that has subscribed to receive those alerts.