Method and apparatus for generating a language independent document abstract

• US 20050119873A1
• Filed: 12/21/2004
• Published: 06/02/2005
• Est. Priority Date: 09/28/1998
• Status: Active Grant

• Current Assignee: Schukhaus Group GmbH LLC (Intellectual Ventures LLC)
• Original Assignee: Intesoft Systems, LLC
• Inventors: Chaney, Garnet R., Rubinstein, Seymour I., Richardson, Robert F.
• Granted Patent: US 7,447,626 B2
• Time in Patent Office: Days
• US Class Current: 704/1
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93

Abstract

A method of extracting significant phrases from one or more documents stored in a computer-readable medium. A sequence of words is read from the one or more documents and a score is determined for each word in the sequence based on the length of the word. The score for each word in the sequence is compared against a threshold score. The sequence of words is indicated to be a significant phrase if the number of words in the sequences that have a score greater than the threshold score equals or exceeds a predetermined number. A sentence containing the sequence of words is retrieved from the document, if the sequence of words is a significant phrase. An abstract of the document is searched to determine if the sentence has been previously included in the abstract. If not, the sentence is added to the abstract.

Claims

1. A method of identifying a significant phrase in a document, the method comprising:

reading a sequence of words from the document;

determining a score for each word in the sequence based on the length of each word;

comparing the score for each word in the sequence against a threshold score;

indicating that the sequence of words is a significant phrase if the number of words in the sequence that have the score greater than the threshold score equals or exceeds a predetermined number;

retrieving a sentence from the document, the sentence containing the sequence of words, if the sequence of words is a significant phrase; and

searching an abstract of the document to determine whether the sentence is included in the abstract.

2. The method of claim 1, wherein reading the sequence of words in the document further comprises reading the sequence of words in the document until a phrase delimiter is detected.

3. The method of claim 1, wherein the score for the word is proportional to the number of characters in the word.

4. The method of claim 3, wherein the number of characters in the word includes the number of numeric digits in the word.

5. The method of claim 1, wherein the score for the word is increased for each capitalized letter in the word.

6. The method of claim 1, wherein the score for the word is the length of the word plus the number of capitalized letters in the word.

7. The method of claim 1, wherein determining the score for each word in the sequence further includes:

determining whether the word exists in a predetermined table; and

retrieving the score for the word from the predetermined table.

8. The method of claim 1, wherein the threshold score is based on an average of the scores of words in the sequence of words.

9. The method of claim 8, wherein the threshold score is adjusted by a predetermined value to be different from the average of the scores of words in the sequence of words.

10. The method according to claim 1, further comprising including the sentence in the abstract, if the sentence is not included in the abstract.

11. The method according to claim 1, further comprising discarding the sentence if the sentence is included in the abstract.

12. The method according to claim 1, wherein the abstract is language independent.

13. A method of identifying a significant phrase in a document, the method comprising:

reading a sequence of words from the document;

determining a score for each word in the sequence based on the length of each word;

comparing the score for each word in the sequence against a threshold score;

indicating that the sequence of words is a significant phrase if the number of words in the sequence that have the score greater than the threshold score equals or exceeds a predetermined number;

storing the sequence of words and the number of words in the sequence, if the sequence of words is a significant phrase.

14. The method according to claim 13, further comprising:

retrieving a sentence from the document, the sentence containing the sequence of words, if the sequence of words is a significant phrase; and

searching an abstract of the document to determine whether the sentence is included in the abstract.

15. The method according to claim 14, further comprising including the sentence in the abstract, if the sentence is not included in the abstract.

16. The method according to claim 14, further comprising discarding the sentence if the sentence is included in the abstract.

17. The method according to claim 14, wherein the abstract is language independent.

18. A computer readable medium containing executable instructions which, when executed in a processing system, cause the system to perform a method for identifying a significant phrase in a document, the method comprising:

reading a sequence of words from the document;

determining a score for each word in the sequence based on the length of each word;

comparing the score for each word in the sequence against a threshold score;

indicating that the sequence of words is a significant phrase if the number of words in the sequence that have the score greater than the threshold score equals or exceeds a predetermined number;

retrieving a sentence from the document, the sentence containing the sequence of words, if the sequence of words is a significant phrase; and

searching an abstract of the document to determine whether the sentence is included in the abstract.

19. The computer readable medium according to claim 18, wherein reading the sequence of words in the document further comprises reading the sequence of words in the document until a phrase delimiter is detected.

20. The computer readable medium according to claim 18, wherein the score for the word is proportional to the number of characters in the word.

21. The computer readable medium according to claim 20, wherein the number of characters in the word includes the number of numeric digits in the word.

22. The computer readable medium according to claim 18, wherein the score for the word is increased for each capitalized letter in the word.

23. The computer readable medium according to claim 18, wherein the score for the word is the length of the word plus the number of capitalized letters in the word.

24. The computer readable medium according to claim 18, wherein determining the score for each word in the sequence further includes:

determining whether the word exists in a predetermined table; and

retrieving the score for the word from the predetermined table.

25. The computer readable medium according to claim 18, wherein the threshold score is based on an average of the scores of words in the sequence of words.

26. The computer readable medium according to claim 25, wherein the threshold score is adjusted by a predetermined value to be different from the average of the scores of words in the sequence of words.

27. The computer readable medium according to claim 18, further comprising including the sentence in the abstract, if the sentence is not included in the abstract.

28. The computer readable medium according to claim 18, further comprising discarding the sentence if the sentence is included in the abstract.

29. The computer readable medium according to claim 18, wherein the abstract is language independent.

30. A computer readable medium containing executable instructions which, when executed in a processing system, cause the system to perform a method for identifying a significant phrase in a document, the method comprising:

reading a sequence of words from the document;

determining a score for each word in the sequence based on the length of each word;

comparing the score for each word in the sequence against a threshold score;

indicating that the sequence of words is a significant phrase if the number of words in the sequence that have the score greater than the threshold score equals or exceeds a predetermined number;

storing the sequence of words and the number of words in the sequence, if the sequence of words is a significant phrase.

31. The computer readable medium according to claim 30, wherein the method further comprises:

retrieving a sentence from the document, the sentence containing the sequence of words, if the sequence of words is a significant phrase; and

searching an abstract of the document to determine whether the sentence is included in the abstract.

32. The computer readable medium according to claim 31, wherein the method further comprises including the sentence in the abstract, if the sentence is not included in the abstract.

33. The computer readable medium according to claim 31, wherein the method further comprises discarding the sentence if the sentence is included in the abstract.

34. The computer readable medium according to claim 31, wherein the abstract is language independent.