Generating and using temporal data partition revisions

• US 9,600,526 B2
• Filed: 12/06/2012
• Issued: 03/21/2017
• Est. Priority Date: 12/06/2012
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Johnson, Theodore, Hadjieleftheriou, Marios, Shkapenyuk, Vladislav
• Primary Examiner(s): ASPINWALL, EVAN S
• Application Number:
  US13/706,893
• Publication Number:
  US 20140164409A1
• Time in Patent Office: 1,566 Days
• Field of Search: 707/756
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/24561 Intermediate data storage techniques for performance improvement

Abstract

Concepts and technologies are disclosed herein for generating and using temporal data partition revisions. In some embodiments, a server computer can execute a data management application. The server computer can obtain a data partition associated with a data stream. The data partition can include a data point and a timestamp. The server computer can analyze the timestamp to determine if the data partition corresponds to an update of a base data record. If the server computer determines that the data corresponds to an update, the server computer can compute a revision comprising a delta and a timestamp and store the revision.

Claims

1. A method comprising:

obtaining, at a server computer that executes a data management application, a data partition included in a data stream that is generated by a data source and streamed by the data source, the data partition comprising a single temporal release of a portion of a data table, wherein the portion of the data table is released multiple times over multiple temporal releases, and wherein the data partition comprises a record comprising a data point and a timestamp that indicates a time at which the data point was generated by the data source;

analyzing, by the server computer, contents of the data partition, a header associated with the data partition, and base data stored at a data store to determine whether the data partition corresponds to a new partition or an update of a base data partition stored at the data store, the base data partition comprising a base data point and a base data timestamp associated with the base data;

if a determination is made that the data partition corresponds to the new partition, storing, by the server computer, the new partition as part of the base data at the data store;

if a determination is made that the data partition corresponds to the update, computing, by the server computer, a revision comprising

a revision identifier associated with the revision,

a delta that defines a change made to the base data point by the revision, and

the timestamp that indicates the time at which the data point was generated by the data source; and

storing, by the server computer, the revision as revision data that is stored separately from the base data partition.

2. The method of claim 1, wherein the base data partition stores a base data identifier, and wherein the revision identifier and the base data identifier are the same.

3. The method of claim 1, wherein the data partition comprises a data identifier, wherein the base data partition stores a base data identifier, and wherein determining that the data partition corresponds to the update comprises determining that the data identifier is the same as the base data identifier.

4. The method of claim 1, further comprising:

receiving a query;

executing the query against the base data; and

storing a result set comprising results of the query against the base data, the result set comprising the base data partition.

5. The method of claim 4, further comprising:

determining that the revision is stored;

retrieving the revision;

applying the revision to the base data partition to generate an updated version of the base data partition; and

outputting a result comprising the updated version of the base data partition.

6. The method of claim 1, further comprising:

retrieving two revisions, each revision of the two revisions comprising an update to a data point of the base data partition;

determining, for the two revisions, a most recent timestamp;

merging the two revisions to generate a merged revision comprising a combined update to a value of the data point; and

storing the merged revision with the updated data point value and the most recent timestamp.

7. The method of claim 6, wherein retrieving the two revisions comprises identifying a base data identifier associated with the base data, and retrieving each of the two revisions based upon determining that each of the two revisions comprises the revision identifier that is the same as the base data identifier.

8. The method of claim 1, wherein determining whether the data partition corresponds to the new partition further comprises:

analyzing data point values associated with contents of the data partition; and

determining, based on the analyzing, if the data partition is related to the base data.

9. The method of claim 4, wherein the query comprises a standing query that is executed periodically by the server computer against the base data.

10. A system comprising:

a processor; and

a memory that stores computer-executable instructions that, when executed by the processor, cause the processor to perform operations comprising

obtaining a data partition included in a data stream that is generated by a data source and streamed by the data source, the data partition comprising a single temporal release of a portion of a data table, wherein the portion of the data table is released multiple times over multiple temporal releases, and wherein the data partition comprises a record comprising a data point and a timestamp that indicates a time at which the data point was generated by the data source,

analyzing contents of the data partition, a header associated with the data partition, and base data stored at a data store to determine whether the data partition corresponds to a new partition or an update of a base data partition stored at the data store, the base data partition comprising a base data point and a base data timestamp associated with the base data,

if a determination is made that the data partition corresponds to the new partition, storing the new partition as part of the base data at the data store,

if a determination is made that the data partition corresponds to the update, computing a revision comprising

a revision identifier associated with the revision,

a delta that defines a change made to the base data point by the revision, and

the timestamp that indicates the time at which the data point was generated by the data source, and

storing the revision as revision data that is stored separately separate from the base data partition.

11. The system of claim 10, wherein the computer-executable instructions, when executed by the processor, cause the processor to perform operations further comprising:

receiving a query;

executing the query against the base data; and

storing a result set comprising results of the query against the base data, the result set comprising the base data partition.

12. The system of claim 11, wherein the computer-executable instructions, when executed by the processor, cause the processor to perform operations further comprising:

determining that the revision is stored;

retrieving the revision;

applying the revision to the base data partition to generate an updated base data partition; and

outputting a result comprising the updated base data partition.

13. The system of claim 10, wherein the computer-executable instructions, when executed by the processor, cause the processor to perform operations further comprising:

retrieving two revisions, each revision of the two revisions comprising an update to a data point of the base data partition;

determining, for the two revisions, a most recent timestamp;

merging the two revisions to generate a merged revision comprising a combined update to a value of the data point; and

storing the merged revision with the updated data point value and the most recent timestamp.

14. The system of claim 10, wherein determining whether the data partition corresponds to the new partition further comprises:

analyzing data point values associated with contents of the data partition; and

determining, based on the analyzing, if the data partition is related to the base data.

15. A computer storage medium having computer-executable instructions stored thereon that, when executed by a processor, cause the processor to perform operations comprising:

obtaining a data partition included in a data stream that is generated by a data source and streamed by the data source, the data partition comprising a single temporal release of a portion of a data table, wherein the portion of the data table is released multiple times over multiple temporal releases, and wherein the data partition comprises a record comprising a data point and a timestamp that indicates a time at which the data point was generated by the data source;

analyzing contents of the data partition, a header associated with the data partition, and base data stored at a data store to determine whether the data partition corresponds to a new partition or an update of a base data partition stored at the data store, the base data partition comprising a base data point and a base data timestamp associated with the base data;

if a determination is made that the data partition corresponds to the new partition, storing the new partition as part of the base data at the data store;

if a determination is made that the data partition corresponds to the update, computing a revision comprising

a revision identifier associated with the revision,

a delta that defines a change made to the base data point by the revision, and

the timestamp that indicates the time at which the data point was generated by the data source; and

storing the revision as revision data that is stored separately from the base data partition.

16. The computer storage medium of claim 15, wherein the computer-executable instructions, when executed by the processor, cause the processor to perform operations further comprising:

retrieving two revisions, each revision of the two revisions comprising an update to a data point of the base data partition;

determining, for the two revisions, a most recent timestamp;

merging the two revisions to generate a merged revision comprising a combined update to a value of the data point;

storing the merged revision with the updated data point value and the most recent timestamp.

17. The computer storage medium of claim 15, wherein the computer-executable instructions, when executed by the processor, cause the processor to perform operations further comprising:

receiving a query;

executing the query against the base data; and

storing a result set comprising results of the query against the base data, the result set comprising the base data partition.

18. The computer storage medium of claim 17, wherein the computer-executable instructions, when executed by the processor, cause the processor to perform operations further comprising:

determining that the revision is stored;

retrieving the revision;

applying the revision to the base data partition to generate an updated base data partition; and

outputting a result comprising the updated base data partition.

19. The computer storage medium of claim 15, wherein determining whether the data partition corresponds to the new partition further comprises:

analyzing data point values associated with contents of the data partition; and

determining, based on the analyzing, if the data partition is related to the base data.