Utilizing classification and text analytics for annotating documents to allow quick scanning

• US 10,430,506 B2
• Filed: 12/10/2012
• Issued: 10/01/2019
• Est. Priority Date: 12/10/2012
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Emanuel, Barton W., Paulis, Mark W., Roboff, Mark L.
• Primary Examiner(s): Lee, Justin S
• Application Number:
  US13/710,025
• Publication Number:
  US 20140164899A1
• Time in Patent Office: Days
• Field of Search: 704 10, 704 7, 704 1, 704 9, 707708
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G06F 40/268 Morphological analysis
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

Classification, text analytics, and natural language processing are used to evaluate passages, extract text, identify concepts, and provide visual cues and notations to assist readers in scanning and evaluating large amounts of information in a document.

Claims

1. A system for annotating a document stored on a non-transitory computer readable storage medium of a computer comprising:

a classifier executed by the computer, wherein the classifier;

(i) determines a type of the document, wherein the type of document is a format within which the document is written;

(ii) determines a subject domain of the document, wherein the subject domain is a field, topic or genre of content set forth in the document; and

(iii) segments the document into one or more paragraphs and sections based on the document'"'"'s structure;

(b) an annotation model, executed by the computer, with information to determine and drive an annotation strategy based on the type of the document, wherein the annotation strategy specifies one or more annotations to utilize and a location in the document'"'"'s original text and on a margin of the document for the one or more annotations;

(c) a text analytics system, executed by the computer, wherein the subject domain determines which domain model to load into the text analytics system and the domain model identifies terms, phrases, entities, and concepts of the subject domain to be annotated in the document, and wherein the text analytics system;

(i) provides the one or more annotations in the document'"'"'s original text and on the margin of the document for the paragraphs of the document based on the domain model and the annotation model, wherein the one or more annotations comprise domain-specific keywords and concepts; and

(ii) aggregates the one or more annotations in the margin of the document for the paragraphs of the document into one or more section-level aggregated annotations for the sections of the document based on the annotation model; and

(d) a custom viewer/renderer application, executed by the computer, that annotates the document with the one or more annotations in the document'"'"'s original text and on the margin of the document for the paragraphs, and with the one or more section-level aggregated annotations for the sections of the document, and renders the document including the one or more annotations and the one or more section-level aggregated annotations.

2. The system of claim 1, wherein the one or more annotations for the paragraphs of the document are selected from the group consisting of:

parts of speech, phrases, proper names, dates, and relationships.

3. The system of claim 1, wherein the text analytics system further normalizes the one or more annotations for the paragraphs of the document by:

identifying synsets; and

determining normal forms using a lexical ontology.

4. The system of claim 1, wherein the text analytics system further:

removes stop words; and

removes words frequently occurring throughout the document, the subject domain, or a set of documents of which the document is a member.

5. The system of claim 1, wherein the custom viewer/renderer further annotates the document'"'"'s original text by highlighting.

6. The system of claim 1, wherein the custom viewer/renderer saves the rendering to disk.

7. The system of claim 1, wherein the custom viewer/renderer:

generates an extensible markup language (XML) file of the one or more annotations; and

executes in standalone mode based on the XML file without having to access a network to perform classification or text analytics.

14. The system of claim 1, wherein the one or more annotations to utilize is selected from the group consisting of:

word, phrase, concept cloud, list, quality indicator, and timeline.

8. A computer program product for annotating a document, the computer program product comprising:

a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising;

computer readable program code configured to obtain the document;

computer readable program code configured to determine a type of the document, wherein the type of document is a format within which the document is written;

computer readable program code configured to determine a subject domain of the document, wherein the subject domain is a field, topic or genre of content set forth in the document;

computer readable program code configured to segment the document into paragraphs and sections based on the document'"'"'s structure;

computer readable program code configured to determine and drive an annotation strategy based on the type of document using information from an annotation model, wherein the annotation strategy specifies one or more annotations to utilize and a location in the document'"'"'s original text and on a margin of the document for the one or more annotations;

computer readable program code configured to determine a domain model to load based on the subject domain, wherein the domain model identifies terms, phrases, entities, and concepts of the subject domain to be annotated in the document;

computer readable program code configured to provide the one or more annotations in the document'"'"'s original text and on the margin of the document for the paragraphs of the document based on the domain model and annotation model, wherein the one or more annotations comprise domain-specific keywords and concepts;

computer readable program code configured to aggregate the one or more annotations for the paragraphs of the document into one or more section-level aggregated annotations for the sections of the document based on the annotation model;

computer readable program code configured to annotate the document with the one or more annotations in the document'"'"'s original text and on the margin of the document for the paragraphs, and with the one or more section-level aggregated annotations for the sections of the document; and

computer readable program code configured to render the document including the one or more annotations and the one or more section-level aggregated annotations.

9. The computer program product of claim 8, wherein the one or more annotations for the paragraphs of the document are selected from the group consisting of:

parts of speech, phrases, proper names, dates, and relationships.

10. The computer program product of claim 8, further comprising computer readable program code configured to normalize the one or more annotations for the paragraphs of the document by:

identifying synsets; and

determining normal forms using a lexical ontology.

11. The computer program product of claim 8, wherein the annotating further comprises highlighting the document'"'"'s original text.

12. The computer program product of claim 8, further comprising computer readable program code configured to save the rendering to disk.

13. The computer program product of claim 8, further comprising computer readable program code configured to generate an extensible markup language (XML) file of the one or more annotations, wherein a renderer is configured to execute in standalone mode based on the XML file without having to access a network to perform classification or text analytics.

15. The computer program product of claim 8, wherein the one or more annotations to utilize is selected from the group consisting of:

word, phrase, concept cloud, list, quality indicator, and timeline.