×

Verifying data consistency

  • US 10,176,216 B2
  • Filed: 02/01/2016
  • Issued: 01/08/2019
  • Est. Priority Date: 02/01/2016
  • Status: Active Grant
First Claim
Patent Images

1. A method for verifying data consistency between update-in-place data structures and append-only data structures containing change histories associated with the update-in-place data structures, the method comprising:

  • loading data from a first update-in-place data structure to a first set of hash buckets in a processing platform, wherein the data from the first update-in-place data structure comprises a first set of key values that corresponds to rows of data in the first update-in-place data structure, and wherein loading the data from the first update-in-place data structure to the first set of hash buckets is based on a first set of hash values associated with the first set of key values;

    loading data from the append-only data structures to a second set of hash buckets in the processing platform;

    performing a bucket-level comparison between the data in the first set of hash buckets and the data in the second set of hash buckets;

    generating an intermediate report based on the bucket-level comparison, wherein generating the intermediate report based on the bucket-level comparison comprises;

    determining an update occurred to the first update-in-place data structure during the bucket-level comparison;

    identifying transient differences between the first update-in-place data structure and the append-only data structures, wherein the transient differences comprise differences caused by either in-flight transactions, by rollback transactions, or by in-flight transactions and by rollback transactions committed at the first update-in-place data structure after loading the data from the first update-in-place data structure to the first set of hash buckets in the processing platform; and

    removing the transient differences from the intermediate report listing differences between the first update-in-place data structure and the append-only data structures; and

    generating a final report based on the intermediate report and removal of the identified transient differences, wherein the final report comprises persistent differences between the first update-in-place data structure and the append-only data structures and omits the identified transient differences removed from the intermediate report, wherein the final report is generated for live comparison of the first update-in-place data structure and the append-only data structures, and wherein the differences are inserted into a second update-in-place data structure that is associated with the first update-in-place data structure.

View all claims
  • 1 Assignment
Timeline View
Assignment View
    ×
    ×