Topic discovery, summary generation, automatic tagging, and search indexing for segments of a document

• US 9,015,153 B1
• Filed: 03/18/2013
• Issued: 04/21/2015
• Est. Priority Date: 01/29/2010
• Status: Active Grant

• Current Assignee: Linfo IP LLC (Pueblo Nuevo LLC)
• Original Assignee: Guangsheng Zhang
• Inventors: Zhang, Guangsheng
• Primary Examiner(s): GIRMA, ANTENEH B
• Application Number:
  US13/845,087
• Time in Patent Office: 764 Days
• US Class Current: 707/723
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/3346 using probabilistic model
• CPC Class Codes: G06F 16/36 Creation of semantic tools, e.g. ontology or thesauri
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30

Abstract

System and methods are disclosed for discovering topics in sub-segments of documents, and extracting terms from a sub-segment representing topics or summaries of the sub-segment, and displaying such terms in connection with the sub-segment or with the document, which can also function as automatically generated tags or labels for the segments or for the documents. Methods are also disclosed for building search indexes based on specific sub-segments of documents, such that, users can search for contents in a specific segment of the document. One embodiment of such a search index is with emails, blogs, and forum articles that typically contain segmented contents added at different times or by different authors in a format known as a thread, and searching in a specific segment such as the most recently added segment can help quickly find the most relevant information without repeating the same information in other segments in the thread.

Claims

1. A method for extracting and presenting information for document segments, comprising:

obtaining a document, wherein the document is in the form of a communication thread, wherein the document comprises multiple segments, wherein at least two of the segments are created by different persons, wherein the multiple segments are chronologically organized based on the time each segment is added to the document;

identifying the first chronological segment in the document, wherein the first chronological segment contains the content that is most recently added to the document by a specific person in the communication thread, wherein the content of the first chronological segment is a response to the content of a different segment in the document created by a different person at a different time;

dividing the document into a first part and a second part, wherein the first part comprises the first chronological segment, wherein the second part comprises the portion of the document that is not the first chronological segment;

extracting the content of the first part for analysis;

identifying one or more terms in the content of the first part, wherein a term can be a word or phrase or sentence or paragraph;

selecting the one or more terms; and

storing or displaying the selected terms to represent a topic or summary of the document or a chronologically newer content in the document.

2. The method of claim 1, wherein a specific heading is used for displaying the selected terms in a topic field or a subject field in association with the display of the document.

3. The method of claim 1, wherein an existing topic text or subject text based on the content of a chronologically older segment in the document is displayed, the method further comprising:

displaying the selected terms as a new topic text or new subject text to highlight the chronologically newer content in connection with the display of the existing topic text or subject text.

4. The method of claim 3, wherein the new topic text or new subject text is displayed under a new heading, wherein the name of the new heading can at least be “

New Topic”

, “

Current Topic”

, “

New Subject”

, “

Current Subject”

, or contains a term including at least “

latest”

, “

most recent”

, “

current”

or “

new”

to indicate that the topic or subject text is extracted from the chronologically newer content.

5. The method of claim 1, wherein the document has an existing topic text or subject text based on the content of a chronologically older segment, the method further comprising:

updating the existing topic text or subject text based on the selected terms to highlight the chronologically newer content.

6. The method of claim 1, wherein an importance measure is determined for the one or more terms in the first chronological segment, wherein the one or more terms are selected based on the importance measure.

7. The method of claim 6, wherein the importance measure is determined based on the grammatical attribute associated with the term, wherein the grammatical attribute includes at least a subject, a predicate or part of a predicate, a head of a multi-word phrase, a modifier in a multi-word phrase, or a sub-phrase of a multi-word phrase.

8. The method of claim 7, wherein the method of determining the importance measure comprises:

assigning a weighting co-efficient to the term based on the grammatical attribute associated with the term; and

producing the importance measure based at least on the weighting co-efficient.

9. The method of claim 8, wherein the method of determining the importance measure further comprises:

incrementing a cumulative value for the term based on the number of occurrences of the term in the first chronological segment and the weighting co-efficient for each corresponding occurrence of the term, wherein the importance measure is produced further based on the cumulative value.

10. The method of claim 1, wherein the one or more terms are selected further based on the frequency of the one or more terms in the first chronological segment or in the document, or the frequency of the one or more terms in other documents, or a combination thereof.

11. The method of claim 10, wherein the one or more terms are selected further based on the difference between the frequency of the one or more terms in the first chronological segment and the frequency of the one or more terms in other documents.

12. A computer system for discovering and presenting information in document segments, comprising:

a computer processor configured to receive a portion of content extracted from a document segment, and

to enable the display of the portion of the content in a display area, wherein the portion of the content is displayed as a tag text or label text of the document segment, wherein the tag or label text represents a topic or a summary of the document or a chronologically newer segment in the document, wherein the document contains multiple text segments, wherein at least two of the text segments are created by different persons, wherein the multiple segments are chronologically organized based on the time each segment is added to the document, wherein the type of document includes an email or forum discussion or blog or other documents in the format of a communication thread, wherein the portion of the content is identified and extracted by;

(a) identifying the first chronological segment in the document, wherein the first chronological segment contains the content that is most recently added to the document by a specific person in the communication thread, wherein the content of the first chronological segment is a response to the content of a different segment in the document created by a different person at a different time,

(b) dividing the document into a first part and a second part, wherein the first part comprises the first chronological segment, wherein the second part comprises the portion of the document that is not the first chronological segment,

(c) extracting the content of the first part for analysis,

(d) identifying a first term in the content of the first part, wherein the first term includes a word or phrase,

(e) selecting the first term or a sentence or paragraph containing the first term, and

(f) extracting the first term or the sentence or paragraph as the portion of the content.

13. The system of claim 12, wherein a specific heading is used for displaying the portion of the content in a topic field or a subject field in association with the display of the document.

14. The system of claim 12, wherein an existing topic text or subject text based on the content of a chronologically older segment in the document is displayed, wherein the portion of the content is displayed as a new topic text or new subject text to highlight the chronologically newer content in connection with the display of the existing topic text or subject text.

15. The system of claim 14, wherein the new topic text or new subject text is displayed under a new heading, wherein the name of the new heading can at least be “

New Topic”

, “

Current Topic”

, “

New Subject”

, “

Current Subject”

, or contains a term including at least “

latest”

, “

most recent”

, “

current”

or “

new”

to indicate that the topic or subject text is extracted from the chronologically newer content.

16. The system of claim 12, wherein an importance measure is determined for the first term, wherein the first term is selected based on the importance measure.

17. The system of claim 16, wherein the importance measure is determined using a method comprising the steps of:

(a) determining a first value for the first term, wherein the first value can be determined based on the number of occurrences of the first term in the first chronological segment, or the presence of the first term in the first chronological segment;

(b) obtaining a second value for the first term, wherein the second value is based on the number of occurrences of the first term in one or more text objects that are different from the first chronological segment; and

(c) producing a score value based on a measurement of the difference between the first value and the second value, wherein the first value is greater than the second value, wherein the score value is an importance score of the first term.

18. The system of claim 17, wherein the score value is equal to the first value subtracted by the second value, or equal to the square of the first value divided by the sum of the first value and the second value.

19. A computer-assisted method for indexing and searching information in document segments, comprising:

receiving a plurality of documents,

wherein each of the documents is in the form of a communication thread, comprising multiple segments, wherein at least two of the segments are created by different persons, wherein the multiple segments are chronologically organized based on the time each segment is added to the document;

for each of the plurality of documents, identifying the first chronological segment in the document, wherein the first chronological segment contains the content that is most recently added to the document by a specific person in the communication thread, wherein the content of the first chronological segment is a response to the content of a different segment in the document created by a different person at a different time;

dividing the document into a first part and a second part, wherein the first part comprises the first chronological segment, wherein the second part comprises the portion of the document that is not the first chronological segment; and

building a search index based on the terms in the first part in one or more of the plurality of documents.

20. The method of claim 19, further comprising:

receiving a query containing a keyword;

matching a term in the index or in the first chronological segment with the keyword in the query; and

returning, as a search result, one or more documents or document segments associated with the matched term.