Document analyzer and metadata generation and use

• US 7,849,081 B1
• Filed: 11/28/2007
• Issued: 12/07/2010
• Est. Priority Date: 11/28/2007
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Welch, Michael J., Chang, Walter
• Primary Examiner(s): LEWIS, CHERYL RENEA
• Application Number:
  US11/946,671
• Time in Patent Office: 1,105 Days
• Field of Search: 707/727, 707/794, 715/256
• US Class Current: 707/727
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification

Abstract

A document analyzer receives a collection of text-based terms associated with a document. The document analyzer performs a statistical analysis on the text-based terms to identify a distribution of where the text-based terms appear in the document and relative frequency indicating how often the text-based terms appear in the document. The document analyzer utilizes the distribution and relative frequency information derived from the statistical analysis to rank multiple themes associated with the document. For example, a received listing of multiple themes may not be presented in any useful order, although it can be assumed that the themes in the listing are present in the document. Based on application of distribution and relative frequency information derived from the analysis, the document analyzer can identify which themes are most relevant to the document as a whole and/or which of themes correspond to different portions (e.g., pages or sections) of the document.

Claims

1. A method comprising:

receiving a collection of text-based terms associated with a document;

performing a statistical analysis on the text-based terms to identify a distribution and relative frequency of the text-based terms in the document;

receiving a listing of multiple themes associated with the document, the listing of multiple themes being derived as a result of performing a semantic analysis of the document; and

utilizing the distribution and relative frequency information derived from the statistical analysis to rank the multiple themes.

2. The method as in claim 1, wherein performing the statistical analysis on the text-based terms to identify the distribution includes:

for a given text-based term, identifying a location range of where the given text-based term can be found in the document based on detecting a first occurrence of the given text-based term and a last occurrence of the given text-based term in the document.

3. The method as in claim 1, wherein performing the statistical analysis on the text-based terms to identify the distribution includes:

for a given text-based terms of the text-based terms, detecting relative locations where the given text-based term can be found in the document; and

based on the relative locations where the given text-based term can be found in the document, generating a weighted average location value specifying a centroid associated with occurrences of the given text-based term in the document.

4. The method as in claim 3, wherein performing the statistical analysis on the text-based terms to identify the distribution includes:

for a given text-based term, identifying a standard deviation of different locations of the given text-based term in the document relative to the centroid.

5. The method as in claim 1, wherein utilizing the distribution and relative frequency information associated with the statistical analysis to rank the multiple themes includes:

generating a ranking of the multiple themes in an order from themes that most likely represent an essence of the document to themes that less likely represent the essence of the document.

6. The method as in claim 5 further comprising:

distributing the ranked themes to at least one remote process that utilizes the ranked themes to identify relevant advertisements for displaying to a user upon retrieval of the document by the user.

7. The method as in claim 1, wherein utilizing the distribution and relative frequency information associated with the statistical analysis to rank the multiple themes includes:

producing a ranked list of most relevant text-based terms found in the document; and

applying the ranked list of most relevant text-based terms to the multiple themes to generate a ranking of the multiple themes in an order from themes that most likely represent an essence of the document to themes that less likely represent the essence of the document.

8. The method as in claim 7, wherein applying the ranked list of most relevant text-based terms to the multiple themes includes:

selecting a most relevant text-based term from the ranked list;

searching the listing of multiple themes to identify a corresponding theme in the listing that includes the most relevant text-based term; and

assigning the corresponding theme that includes the most relevant text-based term to be the most relevant theme.

9. The method as in claim 8, wherein applying the ranked list of most relevant text-based terms to the multiple themes further includes:

identifying each additional most relevant theme by repeating steps of;

selecting a next most relevant text-based term from the ranked list;

searching the listing of multiple themes to identify a corresponding theme in the listing that includes the next most relevant text-based term; and

assigning the corresponding theme that includes the next most relevant text-based term to be the next most relevant theme.

10. The method as in claim 1 further comprising:

generating document metadata including the ranked themes;

associating the document metadata to the document; and

distributing the document metadata to at least one remote process that utilizes the ranked themes to identify relevant advertisements for displaying to a user upon retrieval of the document by the user.

11. The method as in claim 10 further comprising:

utilizing the statistical analysis to produce a ranked list of most relevant text-based terms found in the document, the most relevant text-based terms representing an essence of the document; and

producing the document metadata to include permutations of top text-based terms in the ranked list of most relevant text-based terms.

12. The method as in claim 1 further comprising:

utilizing the statistical information to identify a corresponding set of at least one most representative text-based term for each page of multiple pages in the document; and

for each of the multiple pages, utilizing the corresponding set of at least one most representative text-based terms to identify a theme associated with a respective page that enables matching of a highly relevant advertisement with the respective page.

13. The method as in claim 12, wherein utilizing the corresponding set of at least one most representative text-based term to identify the theme associated with the respective page includes:

identifying a most representative text-based term for the respective page;

searching the listing of multiple themes to identify a corresponding theme in the listing that includes the most representative text-based term for the respective page; and

assigning the corresponding theme in the listing that includes the most representative text-based term for the respective page to be the corresponding theme for the respective page.

14. A method comprising:

receiving a ranking of text-based terms associated with a document, the ranking being based on a statistical analysis of the text-based terms in the document;

receiving a listing of multiple themes associated with the document, the listing of multiple themes being derived as a result of performing a semantic analysis of the document; and

utilizing the ranking of the text-based terms to rank the multiple themes.

15. The method as in claim 14, wherein the statistical analysis indicates a distribution and relative frequency of where the text-based terms appear in the document.

16. The method as in claim 15, wherein utilizing the ranking of the text-based terms to rank the multiple themes includes:

selecting a text-based term from the ranking of the text-based terms; and

utilizing the text-based term to identify one of the multiple themes for associating with the document.

17. A computer readable medium having computer code thereon, the medium comprising:

instructions for receiving a collection of text-based terms associated with a document;

instructions for performing a statistical analysis on the text-based terms to identify a distribution and relative frequency of the text-based terms in the document;

instructions for receiving a listing of multiple themes associated with the document, the listing of multiple themes being derived as a result of performing a semantic analysis of the document; and

instructions for utilizing the distribution and relative frequency information derived from the statistical analysis to rank the multiple themes.

18. The computer readable medium as in claim 17, wherein the instructions for utilizing the distribution and relative frequency information associated with the statistical analysis to rank the multiple themes includes:

instructions for producing a ranked list of most relevant text-based terms found in the document; and

instructions for applying the ranked list of most relevant text-based terms to the multiple themes to generate a ranking of the multiple themes in an order from themes that most likely represent an essence of the document to themes that less likely represent the essence of the document.

19. The computer readable medium as in claim 18, wherein the instructions for applying the ranked list of most relevant text-based terms to the multiple themes includes:

instructions for selecting a first most relevant text-based term from the ranked list;

instructions for searching the listing of multiple themes to identify a corresponding theme in the listing that includes the first most relevant text-based term; and

instructions for assigning the corresponding theme that includes the first most relevant text-based term to be the first most relevant theme.

20. The computer readable medium as in claim 19, wherein the instructions for applying the ranked list of most relevant text-based terms to the multiple themes further includes:

instructions for selecting a second most relevant text-based term from the ranked list;

instructions for searching the listing of multiple themes to identify a corresponding theme in the listing that includes the second most relevant text-based term; and

instructions for assigning the corresponding theme that includes the second most relevant text-based term to be the second most relevant theme.

21. The computer readable medium as in claim 17 further comprising:

instructions for generating document metadata including the ranked themes;

instructions for associating the document metadata to the document; and

instructions for distributing the document metadata to at least one remote process that utilizes the ranked themes to identify relevant advertisements for displaying to a user upon retrieval of the document by the user.

22. The computer readable medium as in claim 21 further comprising:

instructions for utilizing the statistical analysis to produce a ranked list of most relevant text-based terms found in the document, the most relevant text-based terms representing an essence of the document; and

instructions for producing the document metadata to include permutations of top text-based terms in the ranked list of most relevant text-based terms.

23. The computer readable medium as in claim 17 further comprising:

instructions for utilizing the statistical information to identify a corresponding set of at least one most representative text-based term for each page of multiple pages in the document; and

instructions for utilizing the corresponding set of at least one most representative text-based terms to identify a theme associated with a respective page for each of the multiple pages.

24. The computer readable medium as in claim 23, wherein the instructions for utilizing the corresponding set of at least one most representative text-based term to identify the theme associated with the respective page includes:

instructions for identifying a most representative text-based term for the respective page;

instructions for searching the listing of multiple themes to identify a corresponding theme in the listing that includes the most representative text-based term for the respective page; and

instructions for assigning the corresponding theme in the listing that includes the most representative text-based term for the respective page to be the corresponding theme for the respective page.

25. A computer program product including a computer-readable medium having instructions stored thereon for processing data information, such that the instructions, when carried out by a processing device, enable the processing device to perform the operations of:

receiving a collection of text-based terms associated with a document;

performing a statistical analysis on the text-based terms to identify a distribution and relative frequency of the text-based terms in the document;

receiving a listing of multiple themes associated with the document, the listing of multiple themes being derived as a result of performing a semantic analysis of the document; and

utilizing the distribution and relative frequency information derived from the statistical analysis to rank the multiple themes.

26. A computer system comprising:

a processor;

a memory unit that stores instructions associated with an application executed by the processor; and

an interconnect coupling the processor and the memory unit, enabling the computer system to execute the application and perform operations of;

receiving a collection of text-based terms associated with a document;

performing a statistical analysis on the text-based terms to identify a distribution and relative frequency of the text-based terms in the document;

receiving a listing of multiple themes associated with the document, the listing of multiple themes being derived as a result of performing a semantic analysis of the document; and

utilizing the distribution and relative frequency information derived from the statistical analysis to rank the multiple themes.