Determining veracity of data in a repository using a semantic network

• US 8,108,410 B2
• Filed: 08/06/2008
• Issued: 01/31/2012
• Est. Priority Date: 10/09/2006
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Strosaker, Ann Margaret, Strosaker, Michael Thomas
• Primary Examiner(s): Lie, Angela
• Assistant Examiner(s): MINA, FATIMA P
• Application Number:
  US12/186,634
• Publication Number:
  US 20080294610A1
• Time in Patent Office: 1,273 Days
• Field of Search: 707/730, 707/706, 707/707, 707/749, 707/750
• US Class Current: 707/750
• CPC Class Codes: G06F 16/36 Creation of semantic tools, e.g. ontology or thesauri

Abstract

A mechanism for determining the veracity of data in a repository. Responsive to receiving a search query from a user, a semantic network is created from the documents in the repository. A determination is made as to whether data from a first document in the semantic network conflicts with data from a second document in the semantic network. If a conflict exists, a determination is made as to whether the data from the first document is obsolete in comparison to data from the second document. If the data from the first document is obsolete in comparison to data from the second document, a portion of the first document corresponding to the obsolete data is automatically annotating with the data from the second document to form an annotated first document. A search result list is then provided to the user comprising the second document and the annotated first document.

Claims

1. A computer implemented method for determining veracity of data in documents stored in a repository, the computer implemented method comprising:

creating one or more semantic networks from the documents in the repository, wherein said one or more semantic networks define intra-document and inter-document relationships among terms contained within the documents;

responsive to receiving a search query, identifying one or more semantic networks containing nodes matching one or more terms in the search query;

determining an edge density for each node matching a term in the search query;

calculating a relevancy score for each of the one or more semantic networks based on the edge densities of the nodes matching a term in the search query;

determining a relevancy, to the search query, of a first document associated with the one or more semantic networks based on the relevancy score;

determining if data from the first document in one of the semantic networks conflicts with data from a second document in one of the semantic networks;

responsive to a determination that data from the first document conflicts with data from the second document, determining whether the conflicting data from the first document is obsolete in comparison to data from the second document;

wherein determining if the data from the first document is obsolete further comprises;

comparing search frequency information for the data from the first document against search frequency information for the data from the second document; and

responsive to a determination that the search frequency information for the data from the second document is higher than the search frequency information for the data from the first document, determining that the data from the first document is obsolete in comparison with the data from the second document;

responsive to a determination that the conflicting data from the first document is obsolete in comparison to data from the second document, annotating a portion of the first document corresponding to the obsolete data with the data from the second document to form an annotated first document; and

providing a search result list to the user comprising the second document and the annotated first document.

2. The computer implemented method of claim 1, further comprising:

responsive to a user selecting a link to the annotated first document in the search result list, displaying the annotated first document, wherein the annotated first document comprises the obsolete data and the data from the second document, and wherein an annotation in the first document indicates on the displayed first document that the data from the second document overrides the obsolete data in the first document.

3. The computer implemented method of claim 1, wherein determining if the data from the first document is obsolete further comprises:

comparing update information for the data from the first document against update information for the data from the second document; and

responsive to a determination that the update information for the data from the second document is more recent than the update information for the data from the first document, determining that the data from the first document is obsolete in comparison with the data from the second document.

4. The computer implemented method of claim 1, wherein annotating a portion of the first document corresponding to the obsolete data with the data from the second document to form an annotated first document further comprises:

responsive to serving the first document to the user, injecting markup into the first document to annotate the portion of the first document.

5. The computer implemented method of claim 4, wherein the markup is injected into the first document to annotate the obsolete data in the portion of the first document without modifying remaining portions of the first document.

6. The computer implemented method of claim 1, further comprising:

providing a link in the annotated first document, wherein the link displays a summary of conflicting data in the documents and sources of the conflicting data.

7. The computer implemented method of claim 1, wherein determining if data from a first document in the semantic network conflicts with data from a second document in the semantic network further comprises:

comparing propositions in the semantic network to identify propositions that satisfy a matching threshold; and

determining at least one of the nodes in a matched proposition conflicts with a corresponding node in another of the matched propositions.

8. The computer implemented method of claim 1, wherein annotating the first document further comprises removing the obsolete data in the first document.

9. A non-transitory computer readable storage medium having stored thereon computer-executable instructions for determining veracity of data in documents stored in a repository, said computer-executable instructions performing a method comprising:

creating one or more semantic networks from the documents in the repository, wherein said one or more semantic networks define intra-document and inter-document relationships among terms contained within the documents;

responsive to receiving a search query, identifying one or more semantic networks containing nodes matching one or more terms in the search query;

determining an edge density for each node matching a term in the search query;

calculating a relevancy score for each of the one or more semantic networks based on the edge densities of the nodes matching a term in the search query;

determining a relevancy, to the search query, of a first document associated with the one or more semantic networks based on the relevancy score;

determining if data from the first document in one of the semantic networks conflicts with data from a second document in one of the semantic networks;

determining, in response to a determination that data from the first document conflicts with data from the second document, whether the conflicting data from the first document is obsolete in comparison to data from the second document;

wherein said determining if the data from the first document is obsolete further comprises;

comparing search frequency information for the data from the first document against search frequency information for the data from the second document; and

determining, in response to a determination that the search frequency information for the data from the second document is higher than the search frequency information for the data from the first document, that the data from the first document is obsolete in comparison with the data from the second first document;

annotating, in response to a determination that the conflicting data from the first document is obsolete in comparison to data from the second document, a portion of the first document corresponding to the obsolete data with the data from the second document to form an annotated first document; and

providing a search result list to the user comprising the second document and the annotated first document.

10. The non-transitory computer readable storage medium of claim 9, further comprising:

displaying the annotated first document in response to the user selecting a link to the annotated first document in the search result list, wherein the annotated first document comprises the obsolete data and the data from the second document, and wherein an annotation in the first document indicates on the displayed first document that the data from the second document overrides the obsolete data in the first document.

11. The non-transitory computer readable storage medium of claim 9, wherein said determining if the data from the first document is obsolete further comprises:

comparing update information for the data from the first document against update information for the data from the second document; and

determining, in response to a determination that the update information for the data from the second document is more recent than the update information for the data from the first document, that the data from the first document is obsolete in comparison with the data from the second document.

12. The non-transitory computer readable storage medium of claim 9, wherein said annotating a portion of the first document corresponding to the obsolete data with the data from the second document to form an annotated first document further comprises:

injecting markup into the first document to annotate the portion of the first document in response to serving the first document to the user.

13. The non-transitory computer readable storage medium of claim 12, wherein the markup is injected into the first document to annotate the obsolete data in the portion of the first document without modifying remaining portions of the first document.

14. The non-transitory computer readable storage medium of claim 9, wherein said method further comprises:

providing a link in the annotated first document, wherein the link displays a summary of conflicting data in the documents and sources of the conflicting data.

15. The non-transitory computer readable storage medium of claim 9, wherein said determining if data from a first document in the semantic network conflicts with data from a second document in the semantic network further comprises:

comparing propositions in the semantic network to identify propositions that satisfy a matching threshold; and

determining at least one of the nodes in a matched proposition conflicts with a corresponding node in another of the matched propositions.