Information processing for searching categorizing information in a document based on a categorization hierarchy and extracted phrases

• US 6,571,240 B1
• Filed: 02/02/2000
• Issued: 05/27/2003
• Est. Priority Date: 02/02/2000
• Status: Expired due to Term

• Current Assignee: S Aqua Semiconductor LLC (Intellectual Ventures LLC), ProfessorQ, Inc.
• Original Assignee: IP Learn LLC
• Inventors: Tong, Peter P., Ho, Chi Fai
• Primary Examiner(s): ROBINSON, GRETA LEE
• Application Number:
  US09/496,863
• Time in Patent Office: 1,210 Days
• Field of Search: 707/5, 707/4, 707/3, 707/500, 707/100, 706/47, 704/9
• US Class Current: 1/1
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access

Abstract

Processing automatically information in a document to be incorporated into databases to be searched, retrieved and learned. This would significantly enhance categorizing information in the domain so that information can be systematically and efficiently retrieved when needed. In one approach, first, the context or the domain of the document is determined. Then, domain-specific phrases in the document are automatically extracted based on grammar and dictionaries. From these phrases, categories in a category hierarchy are identified, and the document is linked to those categories. Phrases in the document that cannot be categorized are identified to be analyzed. If these new phrases are relevant, new categories may be created based on suggestions provided to categorize them. Later when a user asks a question that is related to the categorized phrases, the corresponding categories are identified, with the document retrieved to respond to the question. In one approach, the question is in natural-language.

Claims

1. A method to process a document from a Web site, based on a categorization hierarchy which has a plurality of categories, each category including one or more phrases, the method comprising:

extracting phrases from the document;

categorizing at least one of the extracted phrases under a category of the categorization hierarchy; and

identifying at least one of the extracted phrases that cannot be categorized into the categorization hierarchy for analysis;

such that information in the document can be appropriately categorized and the document can be systematically retrieved by a natural language responding engine when needed;

wherein the location of the document is related to a URL;

wherein the document includes at least an image when the document is displayed on the Web site, and the method includes not categorizing the image; and

wherein the document includes at least a phrase that is hidden when the document is displayed on the Web site, and the method includes extracting that phrase for categorizing.

2. A method as recited in claim 1 further comprising determining the domain of the document.

3. A method as recited in claim 1 wherein the document is related to a domain.

4. A method as recited in claim 3 wherein the extracting phrases depends on a negative dictionary such that phrases in the negative dictionary will not be extracted from the document for categorizing.

5. A method as recited in claim 4 wherein the negative dictionary includes phrases that are specific to the domain, and phrases that are not specific to the domain.

6. A method as recited in claim 1 further comprising providing a suggestion on categorizing the phrase that cannot be categorized.

7. A method as recited in claim 1 wherein a new category is created in view of the phrase that cannot be categorized.

8. A method as recited in claim 3 wherein the method enables a user to learn materials related to the domain, and to learn information related to the identified phrases.

9. A method as recited in claim 1 wherein whether the document is retrieved by the natural language responding engine to be presented to a user depends on the source of the document.

10. A method as recited in claim 1 wherein:

a first phrase is related to a first category;

a second phrase is related to a second category; and

if the first phrase precedes the second phrase in normal usage, then the first category and the second category are grouped together in the categorization hierarchy.

11. An apparatus to process a document from a Web site, based on a categorization hierarchy, which has a plurality of categories, each category including one or more phrases, the apparatus comprising:

an extractor configured to extract phrases from the document;

a categorizer configured to categorize at least one of the extracted phrases under a category of the categorization hierarchy; and

an identifier configured to identify at least one of the extracted phrases that cannot be categorized into the categorization hierarchy for analysis;

such that information in the document can be appropriately categorized and the document can be systematically retrieved by a natural language responding engine when needed; and

wherein the location of the document is related to a URL;

wherein the document includes at least an image when the document is displayed on the Web site, and the method includes not categorizing the image; and

wherein the document includes at least a phrase that is hidden when the document is displayed on the Web site, and the method includes extracting that phrase for categorizing.

12. An apparatus as recited in claim 11 further comprising a domain determinator configured to determine the domain of the document.

13. An apparatus as recited in claim 11 wherein the document is related to a domain.

14. An apparatus as recited in claim 13 wherein the extractor uses a negative dictionary such that phrases in the negative dictionary will not be extracted from the document for categorizing.

15. An apparatus as recited in claim 14 wherein the negative dictionary includes phrases that are specific to the domain, and phrases that are not specific to the domain.

16. An apparatus as recited in claim 11 further comprising an identifier configured to provide a suggestion on categorizing the phrase that cannot be categorized.

17. An apparatus as recited in claim 11 wherein a new category is created in view of the phrase that cannot be categorized.

18. An apparatus as recited in claim 13 wherein the apparatus enables a user to learn materials related to the domain, and to learn information related to the identified phrases.

19. An apparatus as recited in claim 11 wherein whether the document is retrieved by the natural language responding engine to be presented to a user depends on the source of the document.

20. An apparatus as recited in claim 11 wherein:

a first phrase is related to a first category;

a second phrase is related to a second category; and

if the first phrase precedes the second phrase in normal usage, then the first category and the second category are grouped together in the categorization hierarchy.