Information retrieval through identification of prominent notions

• US 8,375,033 B2
• Filed: 10/19/2009
• Issued: 02/12/2013
• Est. Priority Date: 10/19/2009
• Status: Expired due to Fees

• Current Assignee: Avraham Shpigel, Shalom Wintner
• Original Assignee: Avraham Shpigel, Shalom Wintner
• Inventors: Shpigel, Avraham, Wintner, Shalom
• Primary Examiner(s): LE, DEBBIE M
• Application Number:
  US12/588,505
• Publication Number:
  US 20110093257A1
• Time in Patent Office: 1,212 Days
• Field of Search: 707/706, 707/736, 707/748, 707/770, 704/9
• US Class Current: 707/748
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars

Abstract

A system and method for information retrieval from a corpus of text based on offline prominent sentences extraction, and online prominent sentences retrieval ordered by predefined criteria, and recommending online cross-interest prominent sentences.

Claims

1. A method for information retrieval, performed by a computer processor (115), the method comprising the steps of:

a. processing a corpus of sentences, wherein said processing comprises the steps of

(i) providing a ranking criteria module (135), containing predefined ranking criteria, and coupled to operate with the computer processor;

(ii) retrieving textual documents from a corpus of text documents (120), wherein said corpus is in operative communication flow with the computer processor;

(iii) extracting prominent sentences from said textual documents, using said ranking criteria module; and

(iv) updating a database of prominent sentences (130), coupled to operate with the computer processor; and

b. retrieving prominent sentences from said database of prominent sentences, prioritized by said predefined ranking criteria and by user search keywords,

wherein said extracting of prominent sentences comprises the step of scoring sentences according to at least one extracted rule; and

wherein said extracted rule comprises the steps of;

A. retrieving multiple sentences from said corpus of text documents, wherein for each of said retrieved sentences perform the following steps;

1) identifying the structure of said retrieved sentence;

2) identifying a verb pattern in said retrieved sentence, using said identified structure; and

3) scoring said identified verb pattern, using said ranking criteria module;

B. summing the score of each uniquely identified verb pattern over all the occurrences of said uniquely identified verb pattern in said corpus, to thereby generate aggregated scores for all of said uniquely identified verb patterns; and

C. matching said aggregated scores of said uniquely identified verb patterns with a pre-defined threshold, to thereby generate prominence assigning rules.

2. The method of claim 1, wherein said step of scoring sentences is further performed according to parameters selected from the group including:

sentence structure, frequency of NP subject, frequency of NP subject head, and a combination thereof.

3. The method of claim 2, wherein said sentence structure comprising from left to right, for left to right writing languages and from right to left for right to left writing languages:

NP subject, verb pattern, text following said verb pattern, dot.

4. The method of claim 1, wherein said scoring of a verb pattern further includes the summing of the scores of at least one of the following parameters:

number of occurrences of the NP subject in the same document both before and after the occurrence of the pattern, number of occurrences of the NP subject head of the subject in the same document both before and after the occurrence of the pattern, length in words of the NP subject, length in words of the verb cluster, length in words of the sentence, sentence tense, sentence aspect, sentence mood, sentence voice of the verbs in the pattern, subject is available in an existing library of concepts, NP subject head is available in an existing library of concepts, and a combination thereof.

5. The method of claim 1, wherein said verb pattern comprising from at least one of:

verb cluster, the determination word of the object followed the verb cluster, punctuation, and a combination thereof.

6. The method of claim 1, wherein said retrieving of prominent sentences prioritized by said predefined ranking criteria comprising from at least one of:

search keywords cited in document, document type, number of search keywords cited in a single prominent sentence, search keywords cited in the subject part of the prominent sentence, formal definition, number of words in subject, number of words in prominent sentence, the score of the prominent sentence, document creation date, frequency of interest in prominent sentence, morphological information, part of speech of a search keyword in prominent sentence, the ratio of all prominent sentences in document to the document length, document text quality, document size (in words) and a combination thereof.

7. The method of claim 1 further comprising the step of recommending online cross-interest prominent sentences, and wherein said recommending cross-interest prominent sentences comprises the steps of:

a. indicating interest in prominent sentences by a user;

b. updating the statistics of cross-interested prominent sentences in a cross-interested prominent sentences module, coupled to operate with the computer processor; and

c. using the updated statistics of cross-interested prominent sentences to recommend at least one cross-interest prominent sentence to at least one user.

8. The method of claim 1, wherein said corpus of text comprises data components selected from the group including:

any single document, any collection of documents, organizational project documents, organizational documents, any vertical corpus of text, corpora, and a combination thereof.

9. The method of claim 1, wherein said processing of a corpus of sentences is performed offline.

10. The method of claim 1, wherein said retrieving of prominent sentences, from said database of prominent sentences, is performed online.

11. A computer software product for searching and retrieving prominent sentences from textual documents, the computer software product embodied in a non-transitory computer-readable medium in which program instructions are stored, which instructions, wherein the program instructions, when read by a computer, perform a method for information retrieval comprising the steps of:

a. providing a ranking criteria module, containing predefined ranking criteria, and coupled to operate with the computer processor;

b. extracting prominent sentences from a textual document, using said ranking criteria module; and

c. retrieving prominent sentences from a textual document, prioritized by said predefined ranking criteria, and by one or more user provided search keywords

wherein said extracting of prominent sentences comprises the step of scoring sentences according to at least one extracted rule; and

wherein said extracted rule comprises the steps of;

A. retrieving multiple sentences from said corpus of text documents, wherein for each of said retrieved sentences perform the following steps;

1) identifying the structure of said retrieved sentence;

2) identifying a verb pattern in said retrieved sentence, using said identified structure; and

3) scoring said identified verb pattern, using said ranking criteria module;

B. summing the score of each uniquely identified verb pattern over all the occurrences of said uniquely identified verb pattern in said corpus, to thereby generate aggregated scores for all of said uniquely identified verb patterns; and

C. matching said aggregated scores of said uniquely identified verb patterns with a pre-defined threshold, to thereby generate prominence assigning rules.

12. The computer software product of claim 11, wherein said step of scoring sentences is further performed according to parameters selected from the group including:

sentence structure, frequency of NP subject, frequency of NP subject head, and a combination thereof.

13. The computer software product of claim 12, wherein said sentence structure comprises from left to right, for left to right writing languages and from right to left for right to left writing languages:

NP subject, verb pattern, text following said verb pattern, dot.

14. The computer software product of claim 11 wherein said scoring of a verb pattern further includes the summing of the scores of at least one of the following parameters:

number of occurrences of the NP subject of the same document both before and after the occurrence of the pattern, number of occurrences of the NP subject head of the subject in the same document both before and after the occurrence of the pattern, length in words of the NP subject, length in words of the verb cluster, length in words of the sentence, sentence tense, sentence aspect, sentence mood, sentence voice of the verbs in the pattern, subject is available in an existing library of concepts, NP subject head is available in an existing library of concepts, and a combination thereof.

15. The computer software product of claim 11 wherein said verb pattern comprising from at least one of:

verb cluster, the determination word of the object followed the verb cluster, punctuation, and a combination thereof.

16. The computer software product of claim 11, wherein said retrieving of prominent sentences prioritized by said predefined ranking criteria comprising from at least one of:

search keywords cited in document, document type, number of search keywords cited in a single prominent sentence, search keywords cited in the subject part of the prominent sentence, formal definition, number of words in subject, number of words in prominent sentence, the score of the prominent sentence, document creation date, frequency of interest in prominent sentence, morphological information, part of speech of a search keyword in prominent sentence, the ratio of all prominent sentences in document to the document length, document text quality, document size (in words) and a combination thereof.

17. The method of claim 11, wherein said processing of a corpus of sentences is performed offline, and wherein said retrieving of prominent sentences, from said database of prominent sentences, is performed online.