Method, system, apparatus, program code and means for determining a redundancy of information

• US 20060004878A1
• Filed: 09/15/2004
• Published: 01/05/2006
• Est. Priority Date: 07/02/2004
• Status: Active Grant

• Current Assignee: Regulatory DataCorp, Inc. (Moody's Corporation)
• Original Assignee: Regulatory DataCorp, Inc. (Moody's Corporation)
• Inventors: MacDonald, Alasdair, Lawrence, David, Nitze, Peter
• Granted Patent: US 8,442,953 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/24578 using ranking

Abstract

Some embodiments include a system, method, apparatus and means for determining that first information in an input data packet is not redundant with second information previously stored in a database system, includes receiving the input data packet, generating a database query based on one or more tagged portions in the input data packet, comparing second information retrieved by the database query with the first information to identify at least a first portion of the first information that is different than the second information, and causing storage of the at least a first portion of the first information in the database system.

Claims

1. A method for determining that first information in an input data packet is not redundant with second information previously stored in a database system, the method comprising:

receiving the input data packet;

generating a database query based on one or more tagged portions in said input data packet;

comparing second information retrieved by said database query with said first information to identify at least a first portion of said first information that is different than said second information; and

causing storage of said at least a first portion of said first information in said database system.

2. The method of claim 1, wherein said tagged portions include tags identifying one or more relevant terms.

3. The method of claim 1, wherein said comparing second information further comprises identifying a second portion of said first information that is similar to said second information.

4. The method of claim 3, further comprising:

removing said second portion from said first information prior to said causing storage of said at least first portion.

5. A system, comprising:

a processor; and

a storage device in communication with said processor and storing instructions adapted to be executed by said processor to;

receive first information that is determined to have a relevancy with respect to risk;

compare the first information to second information stored in a database, wherein the second information is stored in a data structure arranged to facilitate the identification of risk;

identify any portions of the first information that are not redundant with respect to the second information; and

store, in the database, the identified portions of the first information.

6. A computer-readable medium having computer executable software code stored therein, the code comprising:

code to receive first information that is determined to have a relevancy with respect to risk;

code to compare the first information to second information stored in a database, wherein the second information is stored in a data structure arranged to facilitate the identification of risk;

code to identify any portions of the first information that are not redundant with respect to the second information; and

code to store, in the database, the identified portions of the first information.

7. A system, comprising:

an information feed;

a first device to store information; and

a second device in communication with the information feed and the first device, the second device configured to receive first information from the information feed, and to cause non-redundant portions of the first information to be stored in the first device.

8. The system of claim 7, wherein the second device includes:

a component to extract second information from the first device.

9. The system of claim 8, wherein the second device further includes:

a component to compare the first information to the second information to identify portions of the first information that are not redundant.

10. The system of claim 8, wherein the second device further includes:

a component to insert the identified portions of the first information into the first device.

11. The system of claim 7, wherein the first device includes a data structure arranged to facilitate the identification of risk.