Microbatch loading

• US 10,140,318 B1
• Filed: 05/30/2014
• Issued: 11/27/2018
• Est. Priority Date: 07/01/2013
• Status: Active Grant

• Current Assignee: Harris Dawn Holdings Incorporated
• Original Assignee: Allscripts Software LLC (Veradigm, Inc.)
• Inventors: Prilutsky, Roman
• Primary Examiner(s): Leroux, Etienne P
• Application Number:
  US14/292,815
• Time in Patent Office: 1,642 Days
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records

Abstract

A method includes receiving structured clinical data in transactional HL7 messages, aggregating the data in a staging area, and merging only the most updated data into a destination database in ordered, validated, calculated, and manipulated sets.

Claims

1. A method comprising:

(a) electronically receiving, over a network, a plurality of HL7 transactional messages containing patient data for a plurality of patients;

(b) electronically sorting, at one or more electronic devices, the plurality of HL7 transactional messages into a plurality of patient queues, each patient queue representing a queue for HL7 transactional messages containing data for a respective particular patient of the plurality of patients;

(c) electronically validating, at an electronic device, a first plurality of HL7 transactional messages for a first particular patient by evaluating the HL7 transactional messages against a plurality of rules;

(d) electronically aggregating, at an electronic device, the first plurality of HL7 transactional messages for the first particular patient, wherein electronically aggregating, at an electronic device, the first plurality of HL7 transactional messages for the first particular patient comprises determining a plurality of CDC operations;

(e) electronically merging, utilizing one or more processors, information corresponding to the first aggregated HL7 transactional messages for the first particular patient into a centralized data repository, one or more servers comprising the centralized data repository;

(f) electronically validating, at an electronic device, a second plurality of HL7 transactional messages for the first particular patient by evaluating the HL7 transactional messages against a plurality of rules;

(g) electronically aggregating, at an electronic device, the second plurality of HL7 transactional messages for the first particular patient;

(h) electronically merging, utilizing one or more processors, information corresponding to the second aggregated HL7 transactional messages for the first particular patient into the centralized data repository;

(i) electronically validating, at an electronic device, a third plurality of HL7 transactional messages for a second particular patient by evaluating the HL7 transactional messages against a plurality of rules;

(j) electronically aggregating, at an electronic device, the third plurality of HL7 transactional messages for the second particular patient;

(k) electronically merging, utilizing one or more processors, information corresponding to the third aggregated HL7 transactional messages for the second particular patient into the centralized data repository;

(l) attempting to electronically validate, at an electronic device, a fourth plurality of HL7 transactional messages for a third particular patient by evaluating the HL7 transactional messages against a plurality of rules; and

(m) determining, based on evaluation of the fourth plurality of HL7 transactional messages against a plurality of rules, that an error occurred when attempting to validate the fourth plurality of HL7 transactional messages.

2. The method of claim 1, wherein the method further includes accessing, by an electronic device, data from the centralized data repository that was merged into the centralized data repository based on information corresponding to the first aggregated HL7 transactional messages for the first particular patient.

3. The method of claim 1, wherein the method further includes displaying, via an electronic display of an electronic device, data from the centralized data repository that was merged into the centralized data repository based on information corresponding to the first aggregated HL7 transactional messages for the first particular patient.

4. The method of claim 1, wherein electronically receiving, over a network, a plurality of HL7 transactional messages containing patient data for a plurality of patients comprises electronically receiving, over a network at a server, a plurality of HL7 transactional messages.

5. The method of claim 1, wherein the number of messages in the first plurality of HL7 transactional messages for the first particular patient is user configurable.

6. The method of claim 1, wherein the method further includes receiving, at an electronic device, input from a user via one or more input devices corresponding to an indication of a number of transactional messages to include an in aggregated “

bulk”

.

7. The method of claim 1, wherein electronically validating, at an electronic device, a first plurality of HL7 transactional messages for a first particular patient by evaluating the HL7 transactional messages against a plurality of rules comprises evaluating the HL7 transactional messages against a plurality of user-customizable rules.

8. The method of claim 1, wherein the method further includes receiving, at an electronic device, input from a user via one or more input devices corresponding to an indication of a number of rule to utilize in evaluating HL7 transactional messages.

9. The method of claim 1, wherein electronically aggregating, at an electronic device, the first plurality of HL7 transactional messages for the first particular patient comprises electronically merging, utilizing one or more processors and the determined CDC operations, information corresponding to the first aggregated HL7 transactional messages for the first particular patient into the centralized data repository comprises.

10. A method comprising:

(a) electronically receiving, over a network, a plurality of transactional messages containing patient data for a plurality of patients;

(b) electronically sorting, at one or more electronic devices, the plurality of transactional messages into a plurality of patient queues, each patient queue representing a queue for transactional messages containing data for a respective particular patient of the plurality of patients;

(c) electronically validating, at an electronic device, a first plurality of transactional messages for a first particular patient by evaluating the transactional messages against a plurality of rules;

(d) electronically aggregating, at an electronic device, the first plurality of transactional messages for the first particular patient, wherein electronically aggregating, at an electronic device, the first plurality of transactional messages for the first particular patient comprises determining a plurality of CDC operations;

(e) electronically merging, utilizing one or more processors, information corresponding to the first aggregated transactional messages for the first particular patient into a centralized data repository, one or more servers comprising the centralized data repository;

(f) electronically validating, at an electronic device, a second plurality of transactional messages for the first particular patient by evaluating the transactional messages against a plurality of rules;

(g) electronically aggregating, at an electronic device, the second plurality of transactional messages for the first particular patient;

(h) electronically merging, utilizing one or more processors, information corresponding to the second aggregated transactional messages for the first particular patient into the centralized data repository;

(i) electronically validating, at an electronic device, a third plurality of transactional messages for a second particular patient by evaluating the transactional messages against a plurality of rules;

(j) electronically aggregating, at an electronic device, the third plurality of transactional messages for the second particular patient;

(k) electronically merging, utilizing one or more processors, information corresponding to the third aggregated transactional messages for the second particular patient into the centralized data repository;

(l) attempting to electronically validate, at an electronic device, a fourth plurality of transactional messages for a third particular patient by evaluating the transactional messages against a plurality of rules; and

(m) determining, based on evaluation of the fourth plurality of transactional messages against a plurality of rules, that an error occurred when attempting to validate the fourth plurality of transactional messages.

11. The method of claim 10, wherein the method further includes accessing, by an electronic device, data from the centralized data repository that was merged into the centralized data repository based on information corresponding to the first aggregated transactional messages for the first particular patient.

12. The method of claim 10, wherein the method further includes displaying, via an electronic display of an electronic device, data from the centralized data repository that was merged into the centralized data repository based on information corresponding to the first aggregated transactional messages for the first particular patient.

13. The method of claim 10, wherein electronically receiving, over a network, a plurality of transactional messages containing patient data for a plurality of patients comprises electronically receiving, over a network at a server, a plurality of transactional messages.

14. The method of claim 10, wherein the number of messages in the first plurality of transactional messages for the first particular patient is user configurable.

15. The method of claim 10, wherein the method further includes receiving, at an electronic device, input from a user via one or more input devices corresponding to an indication of a number of transactional messages to include an in aggregated “

bulk”

.

16. The method of claim 10, wherein electronically validating, at an electronic device, a first plurality of transactional messages for a first particular patient by evaluating the transactional messages against a plurality of rules comprises evaluating the transactional messages against a plurality of user-customizable rules.

17. The method of claim 10, wherein the method further includes receiving, at an electronic device, input from a user via one or more input devices corresponding to an indication of a number of rule to utilize in evaluating transactional messages.

18. The method of claim 10, wherein electronically aggregating, at an electronic device, the first plurality of transactional messages for the first particular patient further comprises electronically merging, utilizing one or more processors and utilizing the determined CDC operations, information corresponding to the first aggregated transactional messages for the first particular patient into the centralized data repository.

19. In a method in which a large number of electronically receiving HL7 transactional messages containing patient data are electronically loaded into a centralized data repository,

(a) an improvement over a conventional methodology where the received HL7 transactional messages would be loaded serially one at a time into the centralized data repository;

(b) wherein the improvement comprises an improved methodology which includes

(i) continuously electronically sorting the electronically received HL7 transactional messages containing patient data into a plurality of virtual patient queues, each virtual patient queue representing a queue for HL7 transactional messages for a different particular patient,

(ii) loading HL7 transactional messages into the centralized data repository in microbatches by repeatedly, for each respective virtual patient queue containing HL7 transactional messages for that respective particular patient,

(A) identifying, based at least in part on a predetermined chunk size, a respective plurality of HL7 transactional messages for the respective particular patient in the respective virtual patient queue that represent a respective microbatch,

(B) electronically aggregating the respective plurality of HL7 transactional messages identified as representing the respective microbatch, wherein the electronic aggregation involves consolidation of repetitive data such that repetitive data from different aggregated HL7 transactional messages is not reproduced multiple times;

(C) electronically merging, into the centralized data repository, information corresponding to the aggregated HL7 transactional messages identified as representing the respective microbatch;

(c) wherein the improvement represents an improvement to computer functionality in that the improvement reduces an amount of time required to load the large number of electronically received HL7 transactional messages into the centralized data repository as compared to the conventional methodology where the received HL7 transactional messages would be serially loaded one at a time.