Method and apparatus for document filtering capable of efficiently extracting document matching to searcher's intention using learning data

• US 20050065919A1
• Filed: 09/16/2004
• Published: 03/24/2005
• Est. Priority Date: 09/19/2003
• Status: Abandoned Application

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Gotoh, Atsushi, Itoh, Hideo
• Application Number:
  US10/941,835
• Publication Number:
  US 20050065919A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/337 Profile generation, learning or modification

Abstract

A document filtering apparatus includes an information input/output unit, a search word extraction unit, a first ranking search unit, a learning data unit, a classifying parameter generation unit, a second ranking search unit, and a classifying unit. The information input/output unit inputs phrasal information, and outputs search result information. The search word extraction unit extracts a search word from the phrasal information. The first ranking search unit searches a document having the search word from a database, and outputs a first ranking search result. The learning data unit prepares learning data from the first ranking search result. The classifying parameter generation unit generates a classifying parameter from the learning data. The second ranking search unit searches a document having a word corresponding to the classifying parameter from the database. The classifying unit extracts a document matching to a searcher'"'"'s intention, and outputs the document as a second ranking search result.

Claims

1. A document filtering apparatus, comprising:

an information input/output unit configured to input phrasal information, and to output search result information;

a search word extraction unit configured to extract a search word from the phrasal information;

a document ranking search unit configured to perform a first ranking search and a second ranking search, wherein the first ranking search is used to search a database for a document having the search word, and output the document as a first ranking search result;

a learning data generation unit configured to prepare learning data reflecting a searcher'"'"'s intention based on the first ranking search result;

a classifying parameter generation unit configured to generate a classifying parameter from the learning data prepared by the learning data generation unit, the classifying parameter being used by the second ranking search of the document ranking search unit to find a document from the database having a word corresponding to the classifying parameter; and

a classifying unit configured to extract a document matching to the searcher'"'"'s intention, and output the document as a second ranking search result.

2. The document filtering apparatus according to claim 1, wherein the learning data generation unit prepares the learning data using at least a part of the first ranking search result.

3. The document filtering apparatus according to claim 1, wherein the classifying parameter generation unit generates the classifying parameter using a predetermined algorithm.

4. The document filtering apparatus according to claim 3, wherein the predetermined algorithm includes at least one of a linear support vector machine, a Fisher discriminant, and a binary independence model of Bayes.

5. The document filtering apparatus according to claim 1, wherein the classifying unit evaluates documents obtained by the second ranking search, designates the documents as a matched document when a predetermined condition is satisfied and as an unmatched document when the predetermined condition is not satisfied, extracts the matched document, and transmits the matched document to the information input/output unit.

6. The document filtering apparatus according to claim 5, wherein the predetermined condition is calculated using the classifying parameter.

7. The document filtering apparatus according to claim 5, wherein the classifying unit sorts the second ranking search result with a predetermined criterion.

8. The document filtering apparatus according to claim 7, wherein the predetermined criterion includes a score calculation using the classifying parameter.

9. A document filtering apparatus, comprising:

inputting and outputting means for inputting phrasal information, and outputting search result information;

extracting means for extracting a search word from the phrasal information;

document ranking searching means for performing a first ranking search and a second ranking search, wherein the first ranking search searches a database for document having the search word, and outputs the document as a first ranking search result;

preparing means for preparing learning data reflecting a searcher'"'"'s intention based on the first ranking search result;

generating means for generating a classifying parameter from the learning data prepared by the preparing means, the classifying parameter being used by the second ranking search of the document ranking searching means to find a document from the database having a word corresponding to the classifying parameter; and

classifying means for extracting a document matching to the searcher'"'"'s intention, and outputting the document as a second ranking search result.

10. The document filtering apparatus according to claim 9, wherein the preparing means prepares the learning data using at least a part of the first ranking search result.

11. The document filtering apparatus according to claim 9, wherein the generating means generates the classifying parameter using a predetermine algorithm.

12. The document filtering apparatus according to claim 11, wherein the predetermined algorithm includes at least one of a linear support vector machine, a Fisher discriminant, and a binary independence model of Bayes.

13. The document filtering apparatus according to claim 9, wherein the classifying means evaluates documents obtained by the second ranking search, designates the documents as a matched document when a predetermined condition is satisfied and as an unmatched document when the predetermined condition is not satisfied, extracts the matched document, and transmits the matched document to the inputting and outputting

14. The document filtering apparatus according to claim 13, wherein the predetermined condition is calculated using the classifying parameter.

15. The document filtering apparatus according to claim 13, wherein the classifying means sorts the second ranking search result with a predetermined criterion.

16. The document filtering apparatus according to claim 15, wherein the predetermined criterion includes a score calculation using the classifying parameter.

17. A method of document filtering, comprising the steps of:

inputting phrasal information;

extracting a search word from the phrasal information;

searching a database for a document having the search word, and outputting the document as a first ranking search result;

preparing learning data reflecting a searcher'"'"'s intention based on the first ranking search result;

generating a classifying parameter from the learning data prepared by the preparing step;

finding a document from the database, the document containing a word corresponding to the classifying parameter;

picking-up a document matching to the searcher'"'"'s intention;

outputting the document as a second ranking search result; and

displaying the second ranking search result.

18. The method of document filtering according to claim 17, wherein the preparing step prepares the learning data using at least a part of the first ranking search result.

19. The method of document filtering according to claim 17, wherein the generating step generates the classifying parameter using a predetermined algorithm.

20. The method of document filtering according to claim 19, wherein the predetermined algorithm includes at least one of a linear support vector machine, a Fisher discriminant, and a binary independence model of Bayes.

21. The method of document filtering according to claim 17, wherein the classifying step evaluates documents obtained by the second ranking search, designates the documents as a matched document when a predetermined condition is satisfied and as an unmatched document when the predetermined condition is not satisfied, extracts the matched document, and transmits the matched document to the displaying step.

22. The method of document filtering according to claim 21, wherein the predetermined condition is calculated using the classifying parameter.

23. The method of document filtering according to claim 21, wherein the classifying step sorts the second ranking search result with a predetermined criterion.

24. The method of document filtering according to claim 23, wherein the predetermined criterion includes a score calculation using the classifying parameter.

25. A program product for document filtering configured to cause a computer to perform a method of document filtering, the method of document filtering comprising the steps of:

inputting phrasal information;

extracting a search word from the phrasal information;

searching a database for a document having the search word, and outputting the document as a first ranking search result;

preparing learning data reflecting a searcher'"'"'s intention based on the first ranking search result;

generating a classifying parameter from the learning data prepared by the preparing step;

finding a document from the database, the document containing a word corresponding to the classifying parameter;

picking-up a document matching to the searcher'"'"'s intention;

outputting the document as a second ranking search result; and

displaying the second ranking search result.

26. A computer readable medium storing a program product for document filtering configured to cause a computer to perform a method of document filtering, the method of document filtering comprising the steps of:

inputting phrasal information;

extracting a search word from the phrasal information;

searching a database for a document having the search word, and outputting the document as a first ranking search result;

preparing learning data reflecting a searcher'"'"'s intention based on the first ranking search result;

generating a classifying parameter from the learning data prepared by the preparing step;

finding a document from the database, the document containing a word corresponding to the classifying parameter;

picking-up a document matching to the searcher'"'"'s intention;

outputting the document as a second ranking search result; and

displaying the second ranking research result.