Data rationalization

• US 8,515,921 B2
• Filed: 08/03/2010
• Issued: 08/20/2013
• Est. Priority Date: 08/03/2010
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Devine, Roy, Saxena, Vijay, Jaffe, Karen, Triebel, William
• Primary Examiner(s): PHAM, KHANH B
• Application Number:
  US12/849,228
• Publication Number:
  US 20120036170A1
• Time in Patent Office: 1,113 Days
• Field of Search: 707/688, 707/692, 707/694
• US Class Current: 707/692
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination

Abstract

Systems, methods, and other embodiments associated with data rationalization are described. One example method includes receiving data from a primary data repository and automatically rationalizing the data by applying rationalization rules that map one or more non-preferred data instances to a preferred data instance. Any non-preferred data instances that have not been automatically rationalized into a preferred data instance are provided to an interface for manual rationalization. Automatically and manually rationalized data is stored in a rationalized data repository. In addition, rationalization rules based on the manual rationalization are extracted for use in subsequent automatic rationalization operations.

Claims

1. A computer-implemented method, comprising:

receiving a first set of data from a primary data repository;

recording original instances of data from the first set of data;

logically deleting redundant instances of the original instances of data;

linking deleted instances of data to a recorded original instance of data;

recording rationalization rules including a source of the rationalization rules;

automatically rationalizing the first set of data by applying the rationalization rules that map one or more non-preferred data instances to a preferred data instance, where, during the automatic rationalization, a first non-preferred data instance and a first rule used to rationalize the first non-preferred data instance are linked to the first preferred data instance;

storing automatically rationalized data in a rationalized data repository;

providing any non-preferred data instances that have not been automatically rationalized into a preferred data instance to an interface for manual rationalization;

receiving manually rationalized data that has been manually rationalized and storing the manually rationalized data in the rationalized data repository; and

extracting additional rationalization rules based on the manual rationalization for use in subsequent automatic rationalization operations by;

capturing a preferred data instance that is mapped to a given non-preferred data instance during manual mapping;

saving the mapping of the captured data instance to the given non-preferred data instance as an additional rationalization rule to be used in automatic rationalization of subsequently received non-preferred data instances corresponding to the given non-preferred data instance;

recording rationalization rules extracted during manual rationalization and sources of the extracted rules;

receiving a second set of data from the primary data repository; and

automatically rationalizing the second set of data by applying the rationalization rules and the additional rules to the second set of data, where, during the manual rationalization, a second non-preferred data instance and a second rule used to rationalize the second non-preferred data instance are linked to a second preferred data instance.

2. The computer-implemented method of claim 1 further comprising maintaining received data in an original, non-rationalized form in the primary data repository.

3. The computer-implemented method of claim 1 further comprising recording changes made to non-preferred data instances and an identifier that identifies a source of the changes to create an audit trail linking a rationalized data instance to a corresponding parent data instance stored in a primary data repository.

4. The computer-implemented method of claim 1 further comprising converting the received data into XML format prior to performing the automatically rationalizing step and further where the rationalized data repository stores the rationalized data in XML format.

5. The computer-implemented method of claim 1, further comprising eliminating redundant instances of data comprises using a logical delete such that deleted data remains available for subsequent retrieval, maintaining a most recent of the redundant instances of data to be rationalized, and creating a link between the most recent of the redundant instances of data and deleted redundant instances of data.

6. The computer-implemented method of claim 1 where providing any non-preferred data instances to an interface for manual rationalization comprises providing the non-preferred data instances to a web based interface for manual rationalization.

7. The computer-implemented method of claim 1 comprising applying revised rationalization rules to rationalized data in the rationalized data repository and storing the resulting rationalized data in the rationalized data repository.

8. The computer-implemented method of claim 1 further comprising converting the automatically rationalized data and the manually rationalized data stored in the rationalized data repository to a different format and delivering converted rationalized data to a client.

9. A computing system, comprising:

a processor;

a receiving logic configured to cause the processor to at least receive a first set of pharmacovigilance (PV) data and a second set of PV data from a primary data repository;

an auto rationalization logic configured to cause the processor to at least automatically rationalize the first set and the second set of PV data by applying rationalization rules that map one or more non-preferred PV data instances to a preferred PV data instance and store automatically rationalized first set and the second set of PV data instances in a rationalized data repository, where the auto rationalization logic is further configured to link a first non-preferred PV data instance and a first rule used to rationalize the first non-preferred PV data instance to a first preferred PV data instance;

a manual rationalization interface logic configured to cause the processor to at least provide any non-preferred PV data instances to an interface for manual rationalization, receive manually rationalized the first set of PV data, and extract additional rationalization rules based on manual rationalization for use in subsequent automatic rationalization operations of the second set of PV data by;

capturing a preferred data instance that is mapped to a given non-preferred data instance during manual mapping;

saving the mapping of the captured data instance to the given non-preferred data instance as an additional rationalization rule to be used in automatic rationalization of subsequently received non-preferred data instances corresponding to the given non-preferred data instance; and

where the manual rationalization interface logic is further configured to link a second non-preferred PV data instance and a second rule used to rationalize the second non-preferred PV data instance to a second preferred PV data instance; and

a de-duplification logic configured to cause the processor to eliminate redundant instances of data prior to performing automatic rationalization.

10. The computing system of claim 9 further comprising a loading logic configured to cause the processor to least restructures source data according to a predetermined structure without altering a native format of the source data and stores restructured source data in a primary data repository.

11. The computing system of claim 9 further comprising a conversion logic configured to cause the processor to at least convert the received data into an XML format prior to performing automatic rationalization and further where the rationalized data repository stores rationalized data in the XML format.

12. The computing system of claim 9 where the manual rationalization interface logic is configured to cause the processor to provide the non-preferred PV data instances to a web based interface for manual rationalization.

13. The computing system of claim 9 further comprising a publication logic configured to cause the processor to at least convert rationalized data stored in the rationalized data repository to a different format and delivering converted rationalized data to a client.

14. The computing system of claim 9 further comprising an audit logic configured to cause the processor to record changes made to non-preferred data instances and an identifier that identifies a source of the changes to create an audit trail linking a rationalized data instance to a corresponding parent data instance stored in a primary data repository.

15. The computing system of claim 9 where:

the receiving logic comprises means for receiving data;

the auto rationalization logic comprises means for at least automatically rationalizing the data by applying rationalization rules that map one or more non-preferred data instances to a preferred data instance and storing the automatically rationalized data in a rationalized data repository; and

the manual rationalization interface logic comprises means for at least providing any non-preferred data instances that have not been automatically rationalized into a preferred data instance to an interface for manual rationalization, receiving manually rationalized data, storing the manually rationalized data in the rationalized data repository, and extracting rationalization rules based on the manual rationalization for use in subsequent automatic rationalization operations.

16. A non-transitory computer-readable medium storing computer-executable instructions that when executed by a computer cause the computer to perform a method, the method comprising:

receiving a first set of data from a primary data repository;

automatically rationalizing the first set of data by applying rationalization rules that map one or more non-preferred data instances to a preferred data instance, where, during the automatic rationalization, a first non-preferred data instance and a first rule used to rationalize the first non-preferred data instance are linked to the first preferred data instance;

storing automatically rationalized data in a rationalized data repository;

providing any non-preferred data instances that have not been automatically rationalized into a preferred data instance to an interface for manual rationalization;

receiving manually rationalized data and storing the manually rationalized data in the rationalized data repository; and

extracting rationalization rules based on the manual rationalization for use in subsequent automatic rationalization operations by;

capturing a preferred data instance that is mapped to a given non-preferred data instance during manual mapping;

saving the mapping of the captured data instance to the given non-preferred data instance as an additional rationalization rule to be used in automatic rationalization of subsequently received non-preferred data instances corresponding to the given non-preferred data instance;

receiving a second set of data from the primary data repository;

automatically rationalizing the second set of data by applying the rationalization rules and the additional rules to the second set of data, where, during the manual rationalization, a second non-preferred data instance and a second rule used to rationalize the second non-preferred data instance are linked to a second preferred data instance; and

eliminating redundant instances of data by at least using a logical delete such that deleted data remains available for subsequent retrieval, maintaining a most recent of the redundant instances of data to be rationalized, and creating a link between the most recent of the redundant instances of data and deleted redundant instances of data.

17. The computer-readable medium of claim 16 where the method comprises converting the received data into an XML format prior to performing automatic rationalization and further where the rationalized data repository stores the automatically rationalized data and the manually rationalized data in the XML format.