Method and system for determining the focus of a document

• US 20060004752A1
• Filed: 06/23/2005
• Published: 01/05/2006
• Est. Priority Date: 06/30/2004
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Sivan, Ron, Amitay, Einat, Harel, Nadav
• Application Number:
  US11/165,527
• Publication Number:
  US 20060004752A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries

Abstract

A method and system for determining the focus of a document are provided. Candidate topics in the form of topic nodes in a hierarchy of topics are input into a focus determining algorithm. For each candidate topic node, a score is allocated to the topic of each level of the hierarchy of the topic node , the scores for each topic are summed and one or more topics are determined to be the focus of the document based on the scores. The scores allocated to the topic of each parent level of the hierarchy of the topic node are progressively lower for the topic of each parent level of the hierarchy. The candidate topics may be provided by identifying occurrences of references to a topic in a document, providing a plurality of possible topics in the form of topic nodes in a hierarchy of topics, and, for each identified occurrence of a reference to a topic, determining the appropriate topic node and adding the topic node to the candidate topics.

Claims

1. A method for determining the focus of a document, comprising:

providing candidate topics in the form of topic nodes in a hierarchy of topics;

for each candidate topic node, allocating a score to the topic of each level of the hierarchy of the topic node;

summing the scores for each topic; and

determining one or more topics as the focus of the document based on the scores.

2. A method as claimed in claim 1, wherein allocating a score to the topic of each parent level of the hierarchy of the topic node allocates a progressively lower score for the topic of each parent level of the hierarchy of the topic node.

3. A method as claimed in claim 1, wherein the method includes:

identifying occurrences of references to a topic in a document;

providing a plurality of possible topics in the form of topic nodes in a hierarchy of topics;

for each identified occurrence of a reference to a topic, determining the appropriate topic node; and

adding the topic node to the candidate topics.

4. A method as claimed in claim 3, wherein determining the appropriate topic node provides an indication of the level of confidence that the reference relates to the topic node and the allocating a score is based on the level of confidence.

5. A method as claimed in claim 1, wherein determining one of more topics as the focus of the document includes selecting a predetermined number of topics with the highest scores.

6. A method as claimed in claim 1, wherein determining one of more topics as the focus of the document includes selecting topics with a score above a predetermined threshold.

7. A method as claimed in claim 1, wherein determining one of more topics as the focus of the document includes disregarding topics in a hierarchy above or below a topic already selected as a focus.

8. A method as claimed in claim 3, wherein providing a plurality of possible topics in the form of topic nodes in a hierarchy of topics includes providing a list of possible forms of reference for each topic .

9. A method as claimed in claim 3, wherein determining the appropriate topic node includes disambiguating references to a topic.

10. A method as claimed in claim 9, wherein disambiguating references to a topic is carried out by applying heuristics to each reference to a topic including one or more of:

evaluating the words surrounding a reference;

applying additional information stored in relation to predefined references; and

evaluating a context of the reference in the document.

11. A method as claimed in claim 1, wherein the topics are geographic topics and the topic hierarchies include encompassing regions.

12. A system for determining the focus of a document, comprising:

means for providing candidate topics in the form of topic nodes in a hierarchy of topics;

means for allocating a score for each candidate topic node to the topic of each level of the hierarchy of the topic node;

means for summing the scores for each topic; and

means for determining one or more topics as the focus of the document based on the scores.

13. A system as claimed in claim 12, wherein the means for allocating a score to the topic of each parent level of the hierarchy of the topic node allocates a progressively lower score for the topic of each parent level of the hierarchy of the topic node.

14. A system as claimed in claim 12, wherein the system includes:

means for identifying occurrences of references to a topic in a document;

a record of a plurality of possible topics in the form of topic nodes in a hierarchy of topics;

means for determining, for each identified occurrence of a reference to a topic, the appropriate topic node in the record; and

means for adding the topic node to the candidate topics.

15. A system as claimed in claim 14, wherein the means for determining for each identified occurrence of a reference to a topic, the appropriate topic node includes means for providing an indication of the level of confidence that the reference relates to the topic node and the means for allocating a score is based on the level of confidence .

16. A system as claimed in claim 12, wherein the means for determining one of more topics as the focus of the document includes means for selecting a predetermined number of topics with the highest scores.

17. A system as claimed in claim 12, wherein the means for determining one of more topics as the focus of the document includes means for selecting topics with a score above a predetermined threshold.

18. A system as claimed in claim 12, wherein the means for determining one of more topics as the focus of the document includes means for disregarding topics in a hierarchy above or below a topic already selected as a focus.

19. A system as claimed in claim 14, wherein the record of a plurality of possible topics in the form of topic nodes in a hierarchy of topics includes a list of possible forms of reference for each topic.

20. A system as claimed in claim 14, wherein the means for determining, for each identified occurrence of a reference to a topic, the appropriate topic node in the record includes means for disambiguating references to a topic.

21. A system as claimed in claim 20, wherein the means for disambiguating references to a topic applies heuristics to each reference to a topic including one or more of:

evaluating the words surrounding a reference;

applying additional information stored in relation to predefined references; and

evaluating a context of the reference in the document.

22. A system as claimed in claim 12, wherein the topics are geographic topics and the topic hierarchies include encompassing regions.

23. A system as claimed in claim 12, wherein the system is a text mining application and the document is a text document.

24. A system as claimed in claim 23, wherein the document is a web page.

25. A computer program product stored on a computer readable storage medium, comprising computer readable program code means for determining the focus of a document, the code means performing the steps of:

providing candidate topics in the form of topic nodes in a hierarchy of topics;

for each candidate topic node , allocating a score to the topic of each level of the hierarchy of the topic node;

summing the scores for each topic; and

determining one or more topics as the focus of the document based on the scores.