Methods and systems for organizing electronic documents

• US 20040133560A1
• Filed: 01/07/2003
• Published: 07/08/2004
• Est. Priority Date: 01/07/2003
• Status: Abandoned Application

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Simske, Steven J.
• Application Number:
  US10/338,584
• Publication Number:
  US 20040133560A1
• Time in Patent Office: Days
• US Class Current: 707/3
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification

Abstract

A method for organizing electronic documents may include generating a list of weighted keywords for each document, clustering related documents together based on a comparison of the weighted keywords, and linking together portions of documents within a cluster based on a comparison of the weighted keywords.

Claims

1. A method for organizing electronic documents, said method comprising:

generating a list of weighted keywords for one or more documents;

clustering related documents together based on a comparison of said weighted keywords; and

linking together portions of documents within a cluster based on a comparison of said weighted keywords.

2. The method of claim 1, wherein said clustering and said linking of documents are conducted automatically without user input.

3. The method of claim 1, wherein said generating a list of weighted keywords for each document, further comprises conducting zoning analysis on each document to identify a layout of each document.

4. The method of claim 3, wherein said generating a list of weighted keywords for each document further comprises dividing each document into a plurality of files, each file corresponding to a portion of the document as identified by the zoning analysis.

5. A method for generating keywords for a document, said method comprising:

identifying a plurality of words in the document;

identifying a role of each word;

computing a word weight for each word based on the role and position of the word in said document; and

selecting a number of keywords based on computed word weights.

6. The method of claim 5, wherein said identifying a plurality of words in the document comprises analyzing an electronic document and identifying all definable words and numbers.

7. The method of claim 5, wherein said identifying a role of each word, comprises:

lemmatizing the word; and

labeling each word with a corresponding part of speech.

8. The method of claim 7, wherein said labeling each word with a corresponding part of speech, comprises:

identifying an antecedent noun corresponding to each pronoun; and

replacing all pronouns with the corresponding antecedent noun.

9. The method of claim 7, wherein said labeling each word with a corresponding part of speech, further comprises:

identifying and labeling proper nouns;

identifying and labeling common nouns;

distinguishing and labeling singular and plural common nouns; and

identifying and labeling cardinal numbers.

10. The method of claim 7, wherein said labeling each word with a corresponding part of speech, further comprises:

identifying and labeling nouns as subjects of a sentence;

identifying and labeling nouns as objects of a sentence; and

identifying and labeling nouns as other nouns (not subjects or objects) in a sentence.

11. The method of claim 5, wherein said computing a word weight for each word comprises:

counting a number of times that word occurs in the document to produce a word count; and

multiplying said word count by a “

mean role weight” and

a square root of a lemma length.

12. The method of claim 11, wherein said “

mean role weight”

is found by summing an average grammatical role weight, noun role weight, and layout role weight of a word.

13. The method of claim 12, wherein said grammatical role weights, noun role weights, and layout role weights are assigned using a method for determining non-numerical attribute weights.

14. The method of claim 5, wherein said selecting a number of keywords based on word weights, comprises:

ranking the words by their associated word weights; and

selecting a number of words based on word weight to form a keyword list.

15. The method of claim 5, wherein said selecting a number of keywords based on word weight, further comprises generating an extended word set based on selected keywords.

16. A method of generating a summary for documents using weighted keywords from a document keyword list, each keyword having a word weight, said method comprising:

counting a number of keyword occurrences in each sentence;

computing a sentence weight for each sentence based on said number of keyword occurences; and

generating a summary for a document containing one or more of sentences from said document that are selected based on said sentence weights.

17. The method of claim 16, wherein said computing a sentence weight for each sentence comprises summing all said word weights of words in the keyword list found within each sentence.

18. The method of claim 16, wherein said generating a summary containing one or more sentences, comprises:

dividing the sentences into sentence groups; and

including at least one sentence from each sentence group in the summary.

19. The method of claim 18, wherein said sentence groups are paragraphs.

20. The method of claim 16, wherein said generating a summary containing one or more sentences comprises pre-selecting a summary length and including a number of sentences in said summary according to said pre-selected summary length.

21. A method for clustering a plurality of documents, each document having an associated keyword list containing keywords, each keyword having an associated word weight, said method comprising:

locating at least one keyword shared by at least two documents of said plurality of documents;

calculating a shared word weight; and

clustering documents with a shared word weight above a specified threshold.

22. A method for associating at least two text units, each text unit containing one or more weighted keywords, said method comprising:

defining a plurality of text units to compose a corpus of text units;

calculating a text unit relevancy metric for each text unit based on a comparison of said weighted keywords; and

selectively linking text units based on said text unit relevancy metrics.

23. The method of claim 22, wherein said text unit may be a word, phrase, sentence, paragraph, page, or document.

24. The method of claim 22, wherein said selectively linking text units, comprises creating an adaptable link between at least two text units based on said relevancy metrics.

25. The method of claim 24, wherein said adaptable link may be visible or invisible to a user.

26. The method of claim 25, wherein said adaptable link is an Internet hyperlink.

27. A program stored on a medium for storing computer-readable instructions, said program, when executed, causing a host device to:

analyze one or more documents;

generate a list of weighted keywords for each document;

cluster related documents together based on said weighted keywords; and

link together portions of clustered documents based on occurrences of said weighted keywords.

28. The program of claim 27, said program further causing said host device to conduct a zoning analysis on each document to identify the layout of said each document.

29. The program of claim 27, said program further casing said host device to:

recognize a plurality of words in a document;

identify a grammatical role of each recognized word;

compute a word weight for each word based on the grammatical role and position of the word in said document; and

select a number of words as keywords based on the word weights.

30. The program of claim 27, said program further causing the host device to:

lemmatize the words in a document; and

label each word with a corresponding part of speech.

31. The program of claim 27, said program further causing the host device to:

identify an antecedent noun corresponding to each pronoun in a document; and

replace all pronouns with the corresponding antecedent noun.

32. The program of claim 27, said program further causing the host device to calculate a word weight for every term in a document by:

counting a number of times a term occurs in a document; and

multiplying said number of times a term occurs by a “

mean role weight” and

a square root of a lemma length of that term.

33. The program of claim 27, said program further causing the host device to calculate a “

mean role weight”

by summing an average grammatical role weight, noun role weight, and layout role weight of a term.

34. The program of claim 27, said program further causing the host device to calculate grammatical role weights, noun role weights, and layout role weights using a method for weighting non-numerical attributes.

35. The program of claim 27, said program further causing the host device to normalize the words of the keyword list by dividing the word weights in the said keyword list by a highest word weight in the keyword list.

36. The program of claim 27, said program further causing the host device to normalize the words in the keyword list by dividing the word weights in the keyword list by a sum of all word weights in the keyword list.

37. The program of claim 27, said program further causing the host device to generate an extended word set containing selected keywords or selected keywords surrounded by words and phrases.

38. A program stored on a medium for storing computer-readable instructions, said program, when executed, causing a host device to:

count a number of keyword occurrences in each sentence of a document;

compute a sentence weight for each of sentence; and

generate a summary for the document containing one or more sentences from said document based on said sentence weights.

39. The program of claim 38, said program further causing the host device to define a sentence grouping, according to user input, and include at least one sentence in the summary from each sentence group in the sentence grouping.

40. The program of claim 38, said program further causing the host device to create a summary based on a pre-selected user-defined summary length.

41. The program of claim 38, said program further causing the host device to:

locate at least one weighted keyword that is shared among multiple documents or summaries;

calculate a shared word weight; and

cluster documents or summaries with a shared word weight above a specified threshold.

42. The program of claim 38, said program further causing the host device to select a maximum, mean, or minimum shared word weight for clustering based on an average number of keywords shared by the documents or summaries.

43. The program of claim 38, said program further causing the host device to:

define a plurality of text units in a corpus of text units;

calculate a text unit relevancy metric for each text unit based on a comparison of weighted keywords; and

selectively link text units based on the relevancy metrics.

44. The program of claim 38, said program further causing the host device to:

determine a location and a weight of keyword or extended keyword occurrences within a text unit;

calculate a text unit weight based on keyword weights; and

compute a relevancy metric for each text unit by multiplying a weight of a chosen text unit by a sum of other text unit weights divided by respective distances from said chosen text unit.

45. The program of claim 38, said program further causing the host device to create an adaptable link between at least two text units based on relevancy metrics.

46. The program of claim 38, said program further causing the host device to automatically readjust links when new text units are added to the corpus of text units.

47. A system for organizing electronic documents, said system comprising:

means for generating a list of weighted keywords for each document;

means for clustering related documents together based on said weighted keywords; and

means for linking together corresponding portions of said documents within a cluster based on said weighted keywords.

48. The system of claim 47, further comprising means for conducting zoning analysis on each document to identify a layout of the document.

49. The system of claim 47, further comprising means for:

obtaining a plurality of words in a document;

identifying a role of each word;

computing a word weight for each word based on a role and position of the word; and

selecting a number of keywords based on the word weights.

50. The system of claim 47, further comprising means for analyzing electronic documents and identifying all recognizable words and numbers.

51. The system of claim 47, further comprising means for:

lemmatizing words; and

labeling each word in a document with a corresponding part of speech.

52. The system of claim 47, further comprising means for counting the number of times a term occurs in a document and multiplying a term count by a “

mean role weight” and

a square root of a lemma length for that term.

53. The system of claim 47, further comprising means for summing an average grammatical role weight, noun role weight, and layout role weight of a term.

54. The system of claim 47, further comprising means for generating an extended word set containing keywords or keywords surrounded by words and phrases that may supplement a meaning and use of said keywords.

55. The system of claim 47, further comprising means for:

counting a number of keyword occurrences in a sentence;

computing a sentence weight for a sentence based on keyword occurrences; and

generating a summary for a document containing one or more sentences from said document based on sentence weights.

56. The system of claim 47, further comprising means for allowing a user to pre-select a summary length.

57. The system of claim 47, further comprising means for:

locating at least one keyword shared by a plurality of documents;

calculating a shared word weight; and

clustering documents with a shared word weight above a specified threshold.

58. The system of claim 47, further comprising means for:

defining a plurality of text units;

calculating a text unit relevancy metric for each text unit based on a comparison of weighted keywords; and

selectively linking text units based on said relevancy metrics.

59. The system of claim 47, further comprising means for creating an adaptable link between text units based on said relevancy metrics.

60. The system of claim 47, further comprising means for updating links when new documents are added to a previously organized corpus of documents.

61. The system of claim 47, further comprising means for clustering and linking documents without user input.