Aggregation, partitioning, and management of healthcare data for efficient storage and processing

• US 10,373,712 B2
• Filed: 05/08/2015
• Issued: 08/06/2019
• Est. Priority Date: 04/22/2014
• Status: Active Grant

• Current Assignee: Cerner Innovation, Inc. (Oracle Corporation)
• Original Assignee: Cerner Innovation, Inc. (Oracle Corporation)
• Inventors: Whitsitt, Greg, Whitacre, Micah, Olson, Andrew
• Primary Examiner(s): Sorey, Robert A
• Application Number:
  US14/707,627
• Publication Number:
  US 20150302149A1
• Time in Patent Office: 1,551 Days
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records

Abstract

Methods, systems, and computer-readable media are provided for aggregating, partitioning, and storing healthcare data. Healthcare data is collected from various disparate healthcare data sources. The data is aggregated into batches of the same type of data. From here, the data is partitioned according to the data'"'"'s originating healthcare data source. The aggregated and partitioned healthcare data is then stored in a long term storage data store. This system of storing healthcare data allows for efficient retrieval and processing by computing solutions that need access to batches of healthcare data. The system also reduces costs associated with storing data as duplicate storage is eliminated.

Claims

1. A computerized method carried out by at least one server having at least one processor for aggregating, partitioning, and storing raw healthcare data for efficient retrieval, the method comprising:

receiving, at a staging platform associated with a first data center, healthcare data from a plurality of disparate healthcare data sources of a distributed computing environment;

implementing a multi-level partitioning strategy for efficient access to a plurality of healthcare data batches, the multi-level partitioning strategy eliminating the need to process healthcare data that is not required for a computing solution, wherein the multi-level partitioning strategy comprises;

(A) aggregating, using the at least one processor, one or more subsets of the healthcare data having the same file type into batches; and

(B) partitioning the batches according to healthcare data source in order to handle a variety of payload sizes and data types while reducing an amount of storage needed;

storing the healthcare data batches in a long term storage data store associated with a second data center of the plurality of disparate data centers of the distributed computing environment;

receiving a request from a processing node for at least one of the healthcare data batches;

accessing the at least one requested healthcare data batches from the long-term storage data store; and

delivering the at least one requested healthcare data batches, where the at least one requested healthcare data batches is processed to generate clinically relevant outcome information.

2. The method of claim 1, further comprising, prior to aggregating the one or more subsets of healthcare data, filtering and removing one or more pieces of duplicate healthcare data.

3. The method of claim 1, wherein the processing node performs batch processing of data.

4. The method of claim 1, wherein the long-term storage data store stores the healthcare data in a virtual file system.

5. The method of claim 1, further comprising partitioning healthcare data in slices based on a time when the healthcare data was received.

6. The method of claim 5, wherein the slices contain data for one of the following time periods:

one minute,

one hour,

one day,

one week,

one month, or

one year.

7. The method of claim 5, further comprising compacting multiple time slices that have been stored for a given period of time into larger slices to reduce storage space.

8. A computer-implemented system for aggregating, archiving, and managing healthcare data, the computer-implemented system comprising:

one or more disparate originating healthcare data sources;

one or more processing nodes;

a staging component that receives and indexes one or more pieces of healthcare data from a plurality of disparate originating healthcare data sources so that it is optimized for low-latency processing by storing the data in a raw form with additional metadata describing a source and a time received for the data;

an extraction component that extracts one or more pieces of healthcare data received and indexed at the staging component from a plurality of disparate originating healthcare data sources;

an aggregating component that aggregates a subset of the healthcare data extracted by the extraction component, the aggregating component aggregating the subset of healthcare data having the same type into batches;

a partitioning component that sorts the batches by originating healthcare data source; and

a long-term storage data store for storing the batches;

wherein the staging component further receives a request from the one or more processing nodes for at least one of the batches;

wherein the staging component further accesses the at least one of the batches in the long-term storage data store; and

wherein the staging component further delivers the at least one of the batches of healthcare data to the processing node where the at least one of the requested batches will be processed to produce a clinically relevant outcome.

9. The system of claim 8, further comprising a filtering component that filters and removes duplicate pieces of healthcare data.

10. The system of claim 8, wherein the partitioning component further sorts the healthcare data into time slices based on when the healthcare data was received.

11. The system of claim 10, further comprising a compaction component that compacts multiple older time slices into larger slices to reduce needed storage space in the long-term storage data store.

12. The system of claim 8 wherein the plurality of disparate originating healthcare data sources include hospitals, clinics, or health centers.

13. One or more non-transitory computer-storage media having computer-executable instructions embodied thereon that, when executed, perform a method of aggregating, partitioning, and storing healthcare data, the method comprising:

extracting a set of healthcare data from one or more healthcare data sources;

filtering and removing duplicate copies of one or more pieces of the set of healthcare data;

aggregating a subset of the set of healthcare data having the same type;

partitioning the aggregated subset of healthcare data by healthcare data source; and

storing the aggregated and partitioned subset of healthcare data in one or more long-term storage data stores;

receiving a request from a processing node for a batch of healthcare data of a particular type and healthcare data source; and

accessing the batch of healthcare data in the long-term storage data store; and

delivering the batch of healthcare data to the processing node where the requested batch of healthcare data will be processed to produce a clinically relevant outcome.

14. The media of claim 13, further comprising labeling each piece of healthcare data of the set of healthcare data with metadata identifying the time the piece of healthcare data was received and further partitioning the aggregated subset of healthcare data into time slices based on a time the healthcare data was received.

15. The media of claim 14, further comprising compacting older time slices of data into larger slices such that the data requires less physical space for storage.

16. The media of claim 13, wherein the request further specifies a time period for the batch of healthcare data.

17. The method of claim 13, wherein the aggregated and partitioned subset of healthcare data is stored in a hierarchy of folders based on type and healthcare data source.

18. The method of claim 13, wherein the pieces of healthcare data of the set of healthcare data are of varying file sizes.