Selecting tags for a document by analyzing paragraphs of the document

• US 8,280,892 B2
• Filed: 10/01/2008
• Issued: 10/02/2012
• Est. Priority Date: 10/05/2007
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Marvit, David L., Jain, Jawahar, Stergiou, Stergios, Gilman, Alex, Adler, B. Thomas, Sidorowich, John J., Labrou, Yannis
• Primary Examiner(s): Perveen, Rehana
• Assistant Examiner(s): Waldron, Scott A
• Application Number:
  US12/242,984
• Publication Number:
  US 20090094231A1
• Time in Patent Office: 1,462 Days
• Field of Search: None
• US Class Current: 707/749
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates

Abstract

In one embodiment, assigning tags to a document includes accessing the document, where the document comprises text units that include words. The following is performed for each text unit: a subset of words of a text unit is selected as candidate tags, relatedness is established among the candidate tags, and certain candidate tags are selected according to the established relatedness to yield a candidate tag set for the text unit. Relatedness between the candidate tags of each candidate tag set and the candidate tags of other candidate tag sets is determined. At least one candidate tag is assigned to the document according to the determined relatedness.

Claims

1. A computer-implemented method comprising:

accessing a document stored in one or more tangible media, the document comprising a plurality of text units, a text unit comprising a plurality of words, the plurality of words comprising a plurality of keywords;

performing the following for each text unit using a processor;

ranking the plurality of words of the each text unit according to a ranking technique;

selecting one or more highly ranked words as the keywords of the each text unit;

establishing relatedness among the keywords of each text unit; and

selecting one or more keywords according to the established relatedness as one or more candidate tags to yield a candidate tag set for the each text unit;

using the processor, determining relatedness between the candidate tags of each candidate tag set and the candidate tags of other candidate tag sets; and

using the processor, assigning at least one candidate tag to the document according to the determined relatedness.

2. The method of claim 1, the selecting one or more keywords of each text unit according to relatedness further comprising:

clustering the keywords to yield a plurality of clusters; and

designating that the keywords of a cluster are sufficiently related.

3. The method of claim 1, the performing the following for each text unit further comprising:

ranking the keywords according to a ranking technique; and

selecting the highest ranking keyword as a root tag.

4. The method of claim 1, the performing the following for each text unit further comprising:

removing one or more candidate tags that are not sufficiently related to the other candidate tags.

5. The method of claim 1, the determining relatedness between the candidate tags of the each candidate tag set and the candidate tags of the other candidate tag sets further comprising:

generating a profile for a candidate tag of the each candidate tag set that indicates relatedness between the candidate tag and the candidate tags of the other candidate tag sets.

6. The method of claim 1, the determining relatedness between the candidate tags of the each candidate tag set and the candidate tags of the other candidate tag sets further comprising generating a profile for a candidate tag of the each candidate tag set by:

determining the number of candidate tag sets that include the candidate tag; and

generating the profile from the number.

7. The method of claim 1, the determining relatedness between the candidate tags of the each candidate tag set and the candidate tags of the other candidate tag sets further comprising generating a profile for a candidate tag of the each candidate tag set, the candidate tag associated with a weight, the each candidate tag set having a first root tag, by:

performing the following for each other candidate tag set, the each other candidate tag set having a second root tag, to yield a plurality of relatedness values;

establishing an affinity of the second root tag, given the first root tag; and

calculating a relatedness value by multiplying the weight and the affinity; and

generating the profile from the plurality of relatedness values.

8. The method of claim 1, the determining relatedness between the candidate tags of the each candidate tag set and the candidate tags of the other candidate tag sets further comprising generating a profile for a first candidate tag of the each candidate tag set by:

performing the following for each other candidate tag set;

for a second candidate tag of the other candidate tag set, establishing an affinity of the second candidate tag, given the first candidate tag, to yield a plurality of affinities; and

combining the affinities; and

generating the profile from the combined affinities.

9. The method of claim 1, the assigning the at least one candidate tag to the document according to the determined relatedness further comprising:

assigning the at least one candidate tag that is most highly related to the other candidate tags.

10. One or more non-transitory computer-readable tangible media encoding software operable when executed to:

access a document stored in one or more tangible media, the document comprising a plurality of text units, a text unit comprising a plurality of words, the plurality of words comprising a plurality of keywords;

perform the following for each text unit;

rank the plurality of words of the each text unit according to a ranking technique;

select one or more highly ranked words as the keywords of the each text unit;

establish relatedness among the keywords of each text unit; and

select one or more keywords according to the established relatedness as one or more candidate tags to yield a candidate tag set for the each text unit;

determine relatedness between the candidate tags of each candidate tag set and the candidate tags of other candidate tag sets; and

assign at least one candidate tag to the document according to the determined relatedness.

11. The computer-readable tangible media of claim 10, further operable to select one or more keywords of each text unit according to relatedness by:

clustering the keywords to yield a plurality of clusters; and

designating that the keywords of a cluster are sufficiently related.

12. The computer-readable tangible media of claim 10, further operable to perform the following for each text unit by:

ranking the keywords according to a ranking technique; and

selecting the highest ranking keyword as a root tag.

13. The computer-readable tangible media of claim 10, further operable to perform the following for each text unit by:

removing one or more candidate tags that are not sufficiently related to the other candidate tags.

14. The computer-readable tangible media of claim 10, further operable to determine relatedness between the candidate tags of the each candidate tag set and the candidate tags of the other candidate tag sets by:

generating a profile for a candidate tag of the each candidate tag set that indicates relatedness between the candidate tag and the candidate tags of the other candidate tag sets.

15. The computer-readable tangible media of claim 10, further operable to determine relatedness between the candidate tags of the each candidate tag set and the candidate tags of the other candidate tag sets by generating a profile for a candidate tag of the each candidate tag set by:

determining the number of candidate tag sets that include the candidate tag; and

generating the profile from the number.

16. The computer-readable tangible media of claim 10, further operable to determine relatedness between the candidate tags of the each candidate tag set and the candidate tags of the other candidate tag sets by generating a profile for a candidate tag of the each candidate tag set, the candidate tag associated with a weight, the each candidate tag set having a first root tag, by:

performing the following for each other candidate tag set, the each other candidate tag set having a second root tag, to yield a plurality of relatedness values;

establishing an affinity of the second root tag, given the first root tag; and

calculating a relatedness value by multiplying the weight and the affinity; and

generating the profile from the plurality of relatedness values.

17. The computer-readable tangible media of claim 10, further operable to determine relatedness between the candidate tags of the each candidate tag set and the candidate tags of the other candidate tag sets by generating a profile for a first candidate tag of the each candidate tag set by:

performing the following for each other candidate tag set;

for a second candidate tag of the other candidate tag set, establishing an affinity of the second candidate tag, given the first candidate tag, to yield a plurality of affinities; and

combining the affinities; and

generating the profile from the combined affinities.

18. The computer-readable tangible media of claim 10, further operable to assign the at least one candidate tag to the document according to the determined relatedness by:

assigning the at least one candidate tag that is most highly related to the other candidate tags.