Semantic analysis documents to rank terms

• US 20080243820A1
• Filed: 03/27/2007
• Published: 10/02/2008
• Est. Priority Date: 03/27/2007
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Ghamrawi, Nadia, Chang, Walter
• Granted Patent: US 7,873,640 B2
• Time in Patent Office: Days
• US Class Current: 707/5
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0207 Discounts or incentives, e.g. coupons or rebates

Abstract

A method, apparatus and computer program product provides for a semantic analyzer to produce and rank semantic terms to reflect their relationship to the theme and topics of a document. The text and the document can have no relationship to any pre-selected keywords before the semantic analyzer performs text extraction. The semantic analyzer extracts text from a document and performs semantic analysis on the extracted text. The semantic analyzer provides a plurality of ranked semantic terms as a result of the semantic analysis and associates semantic terms with the document as semantic keywords. The semantic terms define content to be presented with the document where the content is an advertisement, a link to a remote information resource or a second document.

Claims

1. A computer-implemented method, comprising:

extracting text from a document;

performing semantic analysis on the text extracted from the document;

providing a plurality of ranked semantic terms as a result of the semantic analysis; and

associating at least one semantic term with the document, the at least one semantic term defining content to be presented with the document.

2. The computer-implemented method as in claim 1, wherein the at least one semantic term defining content to be presented with the document includes defining at least one of the group consisting of an advertisement, a link to a remote information resource and a second document.

3. The computer-implemented method as in claim 1, wherein extracting text from the document includes extracting plain text from the document, the text and the document lacking a relationship to one or more pre-selected keywords.

4. The computer-implemented method as in claim 3, wherein extracting plain text from the document includes:

identifying at least one token within the extracted plain text, the token representing a string of text and characters in the document;

tabulating a token value representing a total number of times the token occurs throughout the document;

identifying at least one term within the extracted plain text, the term comprising a contiguous grouping of tokens;

tabulating a term value (n) for the term (x_j) representing a total number of times the term occurs throughout the document; and

tabulating at least one mention offset for the term, the mention offset (offset(mⁱ)) representing a position of an individual occurrence (m_i) of the term (x_j) in the document within a set of n occurrences of the term, where n can be equal to or greater than 1.

5. The computer-implemented method as in claim 4, further comprising calculating at least one term statistic, the at least one term statistic selected from the group consisting of:

calculating a token frequency (tf(x_j)) for the term as a function of the token values of the tokens in the term, the function comprising at least one of an average and a mean of the token values of the tokens in the term;

calculating a mean offset (moffset(x_j)) for the term; and

calculating an offset standard deviation (soffset(x_j)) for the term.

6. The computer-implemented method as in claim 5, further comprising calculating an article score (ascore(x_j)) when the document is a long article discussing at least one central topic.

7. The computer-implemented method as in claim 5, further comprising calculating a technical article score (tscore(x_j)) when the document is a technical article, the technical article score comprising:

calculating at least one difference (r) between two individual occurrences of the term;

calculating a mean gap (r(x_j)); and

calculating a gap standard deviation (rsdiff(x_j)).

8. The computer-implemented method as in claim 5, further comprising calculating a standard deviation letter score and calculating a micro-frequency letter score when the document is a letter.

9. The computer-implemented method as in claim 1, wherein providing the plurality of ranked semantic terms as a result of the semantic analysis includes:

providing a listing of the plurality of ranked semantic terms, the listing ranking the semantic terms according to one or more term statistics;

selecting one or more semantic terms from the listing to create a preview of the content defined by each of the one or more semantic terms that are selected;

the preview displaying an aggregate of the content defined by each of the one or more semantic terms that are selected; and

designating one or more semantic terms from the listing to be associated with the document as one or more semantic keywords.

10. The computer-implemented method as in claim 1, wherein associating the at least one semantic term into the document includes inserting the at least one semantic term into a metadata portion of the document.

11. The computer-implemented method as in claim 10, wherein inserting the at least one semantic term into the metadata portion of the document includes inserting the at least one semantic term into an XMP (Extensible Metadata Platform) portion of the document.

12. The computer-implemented method as in claim 1, further comprising:

assigning a policy to the document, the policy enabling the document to utilize the at least one semantic term associated with the document as a semantic keyword in order to display the content as the document is presented, the policy further enabling the document to bypass the at least one semantic term associated with the document in order to not display the content as the document is presented.

13. A computer readable medium comprising executable instructions encoded thereon operable on a computerized device to perform processing comprising:

instructions for extracting text from a document;

instructions for performing semantic analysis on the text extracted from the document;

instructions for providing a plurality of ranked semantic terms as a result of the semantic analysis; and

instructions for associating at least one semantic term with the document, the at least one semantic term defining content to be presented with the document.

14. The computer readable medium as in claim 13, wherein the at least one semantic term defining content to be presented with the document include instructions for defining at least one of the group consisting of an advertisement, a link to a remote information resource and a second document.

15. The computer readable medium as in claim 13, wherein instructions for extracting text from the document include instructions for extracting plain text from the document, the text and the document lacking a relationship to one or more pre-selected keywords.

16. The computer readable medium as in claim 15, wherein instructions for extracting plain text from the document include:

instructions for identifying at least one token within the extracted plain text, the token representing a string of text and characters in the document;

instructions for tabulating a token value representing a total number of times the token occurs throughout the document;

instructions for identifying at least one term within the extracted plain text, the term comprising a contiguous grouping of tokens;

instructions for tabulating a term value (n) for the term (x_j) representing a total number of times the term occurs throughout the document; and

instructions for tabulating at least one mention offset for the term, the mention offset (offset(m_i)) representing a position of an individual occurrence (m_i) of the term (x_j) in the document within a set of n occurrences of the term, where n can be equal to or greater than 1.

17. The computer readable medium as in claim 16, further comprising instructions for calculating at least one term statistic, the instructions for calculating at least one term statistic selected from the group consisting of:

instructions for calculating a token frequency (tf(x_j)) for the term as a function of the token values of the tokens in the term, the function comprising at least one of an average and a mean of the token values of the tokens in the term;

instructions for calculating a mean offset (moffset(x_j)) for the term; and

instructions for calculating an offset standard deviation (soffset(x_j)) for the term.

18. The computer readable medium as in claim 17, further comprising instructions for calculating an article score (ascore(x_j)) when the document is a long article discussing at least one central topic.

19. The computer readable medium as in claim 17, further comprising instructions for calculating a technical article score (tscore(x_j)) when the document is a technical article, the instructions for calculating the technical article score comprising:

instructions for calculating at least one difference (r) between two individual occurrences of the term;

instructions for calculating a mean gap (r(x_j)); and

instructions for calculating a gap standard deviation (rsdiff(x_j)).

20. The computer readable medium as in claim 17, further comprising instructions for calculating a standard deviation letter score and instructions for calculating a micro-frequency letter score when the document is a letter.

21. The computer readable medium as in claim 13, wherein instructions for providing the plurality of ranked semantic terms as a result of the semantic analysis includes:

instructions for providing a listing of the plurality of ranked semantic terms, the listing ranking the semantic terms according to one or more term statistics;

instructions for selecting one or more semantic terms from the listing to create a preview of the content defined by each of the one or more semantic terms that are selected;

the preview displaying an aggregate of the content defined by each of the one or more semantic terms that are selected; and

instructions for designating one or more semantic terms from the listing to be associated with the document as one or more semantic keywords.

22. The computer readable medium as in claim 13, wherein instructions for associating the at least one semantic term into the document include instructions for inserting the at least one semantic term into a metadata portion of the document.

23. The computer readable medium as in claim 22, wherein instructions for inserting the at least one semantic term into the metadata portion of the document include instructions for inserting the at least one semantic term into an XMP (Extensible Metadata Platform) portion of the document.

24. The computer readable medium as in claim 13, further comprising:

instructions for assigning a policy to the document, the policy enabling the document to utilize the at least one semantic term associated into the document as a semantic term in order to display the content as the document is presented, the policy further enabling the document to bypass the at least one semantic term associated into the document in order to not display the content as the document is presented.

25. A computer system comprising:

a processor;

a memory unit that stores instructions associated with an application executed by the processor; and

an interconnect coupling the processor and the memory unit, enabling the computer system to execute the application and perform operations of;

extracting text from a document;

performing semantic analysis on the text extracted from the document;

providing a plurality of ranked semantic terms as a result of the semantic analysis; and

associating at least one semantic term with the document, the at least one semantic term defining content to be presented with the document.