Discrepancy curator for documents in a corpus of a cognitive computing system

• US 9,842,161 B2
• Filed: 01/12/2016
• Issued: 12/12/2017
• Est. Priority Date: 01/12/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Byron, Donna K., Feirouz, Elie, Kumar, Ashok, O'Keeffe, William G.
• Primary Examiner(s): Sirjani, Fariba
• Application Number:
  US14/993,538
• Publication Number:
  US 20170199882A1
• Time in Patent Office: 700 Days
• CPC Class Codes: G06F 16/335 Filtering based on additional data, e.g. user or group profiles filtering in web context G06F16/9535, G06F16/9536
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 40/226 Validation
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

Curation of a corpus of a cognitive computing system is performed interactively by reporting on user interface device to a user a parse tree illustration of discrepancies and corresponding assigned confidence factors detected between at least a portion of a first document and a second or more documents in the corpus. Responsive to a user selection of an illustrated discrepancy in the parse tree, a drill-down dialog is prepared and displayed which shows at least a text string for the portion of the first document and at least one conflicting text string from the second or more documents, and which provides at least one user-selectable administrative action option for handling the detected discrepancy. Responsive to receipt of user selection of an administrative action option, the computing system performs the action to handle the detected discrepancy.

Claims

1. A method comprising the steps of:

producing by a computing system a parse tree illustration of one or more discrepancies and confidence factors detected between at least a portion of a first document and a second or more documents in a cognitive computing system corpus, wherein the portion and the second or more documents comprise natural language unstructured data, wherein the parse tree illustration comprises a structure of a cluster of documents encapsulating categorized previously-asked questions about the corpus, and wherein the confidence factors are previously-assigned by a cognitive computing system corresponding to the questions and the documents;

reporting on a user interface device to a user the parse tree illustration;

responsive to receipt by a computing system of a user selection of an illustrated discrepancy in the parse tree, displaying by a computing system on a user interface device to a user a drill-down dialog which shows at least a text string for the portion of the first document and at least one conflicting text string from the second or more documents, and which provides at least one user-selectable administrative action option for handling the detected discrepancy; and

responsive to receipt by a computing system of a user selection of at least one of the administrative action options, performing by the computing system the administrative action, thereby handling the detected discrepancy to curate the corpus.

2. The method as set forth in claim 1 further comprising, responsive to the handling of the discrepancy, recording by the computer the administrative action performed so that the discrepancy may optionally be suppressed from future display or reviewed on the user interface device in the future.

3. The method as set forth in claim 1 wherein the displaying of the drill-down dialog comprises accessing one or more recorded links between the first and the second or more documents and creating a graphical user display according to the links.

4. The method as set forth in claim 1 wherein the discrepancies reflect conflicting answers provided by the first and the second or more documents when queried using a question associated with a category to which the first and second or more documents are assigned.

5. The method as set forth in claim 1 wherein the administrative action options comprise one or more actions selected from the group consisting of flagging the first document, removing the first document, replacing the first document, editing the first document, ignoring the discrepancy, flagging the second or more document, removing the second or more document, replacing the second or more document, and editing the second or more document.

6. The method as set forth in claim 1 wherein at least one of the first, second or more documents comprise a fragment of a document.

7. A computer program product for a user to curate a corpus of a cognitive computing system comprising:

a computer-readable memory device which is not a propagating signal per se; and

program instructions embodied by the computer-readable memory device which, when executed, cause a processor to perform steps of;

producing a parse tree illustration of one or more discrepancies and confidence factors detected between at least a portion of a first document and a second or more documents in a cognitive computing system corpus, wherein the portion and the second or more documents comprise natural language unstructured data, wherein the parse tree illustration comprises a structure of a cluster of documents encapsulating categorized previously-asked questions about the corpus, and wherein the confidence factors are previously-assigned by a cognitive computing system corresponding to the questions and the documents;

reporting on a user interface device to a user the parse tree illustration;

responsive to receipt of a user selection of an illustrated discrepancy in the parse tree, displaying by a computing system on a user interface device to a user a drill-down dialog which shows at least a text string for the portion of the first document and at least one conflicting text string from the second or more documents, and which provides at least one user-selectable administrative action option for handling the detected discrepancy; and

responsive to receipt of a user selection of at least one of the administrative action options, performing by the computing system the administrative action, thereby handling the detected discrepancy to curate the corpus.

8. The computer program product as set forth in claim 7 wherein the program instructions further comprise program instructions for, responsive to the handling of the discrepancy, recording the administrative action performed so that the discrepancy may optionally be suppressed from future display or reviewed on the user interface device in the future.

9. The computer program product as set forth in claim 7 wherein the program instructions for displaying of the drill-down dialog comprises program instructions for accessing one or more recorded links between the first and the second or more documents and creating a graphical user display according to the links.

10. The computer program product as set forth in claim 7 wherein the discrepancies reflect conflicting answers provided by the first and the second or more documents when queried using a question associated with a category to which the first and second or more documents are assigned.

11. The computer program product as set forth in claim 7 wherein the administrative action options comprise one or more actions selected from the group consisting of flagging the first document, removing the first document, replacing the first document, editing the first document, ignoring the discrepancy, flagging the second or more document, removing the second or more document, replacing the second or more document, and editing the second or more document.

12. The computer program product as set forth in claim 7 wherein at least one of the first, second or more documents comprise a fragment of a document.

13. A system for a user to curate a corpus of a cognitive computing system comprising:

a computing system having a processor;

a computer-readable memory device which is not a propagating signal per se; and

program instructions embodied by the computer-readable memory device which, when executed, cause the processor to perform steps of;

producing a parse tree illustration of one or more discrepancies and confidence factors detected between at least a portion of a first document and a second or more documents in a cognitive computing system corpus, wherein the portion and the second or more documents comprise natural language unstructured data, wherein the parse tree illustration comprises a structure of a cluster of documents encapsulating categorized previously-asked questions about the corpus, and wherein the confidence factors are previously-assigned by a cognitive computing system corresponding to the questions and the documents;

reporting on a user interface device to a user the parse tree illustration;

responsive to receipt of a user selection of an illustrated discrepancy in the parse tree, displaying by a computing system on a user interface device to a user a drill-down dialog which shows at least a text string for the portion of the first document and at least one conflicting text string from the second or more documents, and which provides at least one user-selectable administrative action option for handling the detected discrepancy; and

responsive to receipt of a user selection of at least one of the administrative action options, performing by the computing system the administrative action, thereby handling the detected discrepancy to curate the corpus.

14. The system as set forth in claim 13 wherein the program instructions further comprise program instructions for, responsive to the handling of the discrepancy, recording the administrative action performed so that the discrepancy may optionally be suppressed from future display or reviewed on the user interface device in the future.

15. The system as set forth in claim 13 wherein the program instructions for displaying of the drill-down dialog comprises program instructions for accessing one or more recorded links between the first and the second or more documents and creating a graphical user display according to the links.

16. The system as set forth in claim 13 wherein the discrepancies reflect conflicting answers provided by the first and the second or more documents when queried using a question associated with a category to which the first and second or more documents are assigned.

17. The system as set forth in claim 13 wherein the administrative action options comprise one or more actions selected from the group consisting of flagging the first document, removing the first document, replacing the first document, editing the first document, ignoring the discrepancy, flagging the second or more document, removing the second or more document, replacing the second or more document, and editing the second or more document.

18. The system as set forth in claim 13 wherein at least one of the first, second or more documents comprise a fragment of a document.