Keyword extracting device

• US 20020042794A1
• Filed: 09/05/2001
• Published: 04/11/2002
• Est. Priority Date: 01/05/2000
• Status: Active Grant

• Current Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Inventors: Konaka, Hiroki
• Granted Patent: US 7,191,177 B2
• Time in Patent Office: Days
• US Class Current: 707/6
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: Y10S 707/917 Text
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

The object of the present invention is to obtain a keyword extracting device which extracts keywords collectively and efficiently while improving descriptive property and reusability of the information for keyword extracting. A keyword extracting device of the present invention comprises text data input means for inputting a text, pattern processing means for carrying out matching and replacement of a character string based on a pattern in regular expression or its equivalent, pattern storage means having at least a keyword component pattern representing a character string capable of being a component of a keyword, keyword component extracting means for extracting, as keyword components, all character strings which are matched with a keyword component pattern and are not overlapped with each other by using the pattern processing means for a text, keyword candidate set generating means for generating a keyword candidate set from each keyword component, and keyword output means for outputting each keyword candidate of a keyword candidate set as a keyword.

Claims

1. A keyword extracting device comprising:

text data input means for inputting a text,

pattern processing means for carrying out matching and replacement of a character string based on a pattern in regular expression or its equivalent,

pattern storage means having at least a keyword component pattern representing a character string capable of being a component of a keyword,

keyword component extracting means for extracting, as keyword components, all character strings which are matched with the keyword component pattern and are not overlapped with each other by using the pattern processing means for the text,

keyword candidate set generating means for generating a keyword candidate set from each keyword component, and

keyword output means for outputting each keyword candidate of the keyword candidate set as a keyword.

2. A keyword extracting device of claim 1 wherein the keyword candidate set generating means generates a keyword candidate set by exactly using each keyword component as a keyword candidate.

3. A keyword extracting device of claim 1 wherein:

the keyword component extracting means records extracting positions while extracts as keyword components, all character strings which are matched with the keyword component pattern and are not overlapped with each other by using the pattern processing means for the text, and

the keyword candidate set generating means generates the keyword candidate set by coupling, as one keyword candidate, keyword components decided to be continuous from their character string length and extracting position over the text and by exactly using the other a single keyword component as a keyword candidate.

4. A keyword extracting device of claim 3 wherein the keyword candidate set generating means generates the keyword candidate set by using continuous keyword components over the text as a keyword candidate respectively and coupling the preceding keyword component with the following keyword component to make a keyword candidate, while exactly using the other single keyword component as a keyword candidate.

5. A keyword extracting device of claim 3 or 4 wherein the keyword candidate set generating means unifies repeated keyword candidates in a keyword candidate set into one keyword candidate.

6. A keyword extracting device of claim 1, 2, 3, 4 or 5 wherein the pattern storage means has at least the keyword component pattern representing a character string capable of being a component of a keyword and a non-keyword candidate pattern representing a portion which cannot be a candidate of a keyword,

and further comprising a non-keyword removing means for searching all character strings which are matched with the non-keyword candidate pattern and are not overlapped with each other and for replacing the character string with a special character string with which any pattern is never matched by using the pattern processing means for the text.

7. A keyword extracting device of claim 1, 2, 3, 4, 5 or 6 wherein the pattern storage means has at least the keyword component pattern representing a character string capable of being a component of a keyword and an extra keyword component pattern representing a keyword component which is hard to cover with the keyword component pattern,

and further comprising an extra keyword component extracting means for extracting, as extra keyword components, all character strings which are matched with the extra keyword component pattern and are not overlapped with each other, recording an extracting position, and converting each character string matched in the text into a special character string with which any pattern is never matched by using the pattern processing means for the text,

and the keyword candidate set generating means for generating a keyword candidate set from the keyword component and the extra keyword component.

8. A keyword extracting device of claim 1, 2, 3, 4, 5, 6 or 7 wherein the pattern storage means has at least the keyword component pattern representing a character string capable of being a component of a keyword and an unnecessary word pattern representing an unnecessary word,

and further comprising unnecessary word removing means for removing an unnecessary word matched with the unnecessary word pattern from the keyword candidate set by using the pattern processing means for a whole character string of each keyword candidate of the keyword candidate set.

9. A keyword extracting device of claim 8 wherein the pattern storage means has at least the keyword component pattern representing a character string capable of being a component of a keyword, a necessary word pattern representing a necessary word and the unnecessary word pattern representing an unnecessary word, and the unnecessary word removing means removes an unnecessary word which is not matched with the necessary word pattern but is matched with the unnecessary word pattern from the keyword candidate set by using the pattern processing means for a whole character string of each keyword candidate of the keyword candidate set.

10. A keyword extracting device of claim 1, 2, 3, 4, 5, 6, 7, 8 or 9 wherein the pattern storage means has at least the keyword component pattern representing a character string capable of being a component of a keyword and an unnecessary head pattern representing an unnecessary head character string,

and further comprising unnecessary head character string removing means for removing a portion matched with the unnecessary head pattern from a keyword candidate matched with the unnecessary head pattern, by using the pattern processing means for a head character string of each keyword candidate of the keyword candidate set, if this removal results in an empty character string, the original keyword candidate is deleted from a keyword candidate set. If the result of this removal is not an empty character string, the original keyword candidate is replaced by a keyword candidate in which an unnecessary head character string is removed.

11. A keyword extracting device of claim 10 wherein the pattern storage means has at least the keyword component pattern representing a character string capable of being a component of a keyword, an unnecessary head pattern representing the unnecessary head character string and the necessary head pattern representing a necessary head character string, and unnecessary head character string removing means removes a portion matched with the unnecessary head pattern from a keyword candidate which is not matched with the necessary head pattern but is matched with the unnecessary head pattern, by using the pattern processing means for a head character string of each keyword candidate of the keyword candidate set, if this removal results in an empty character string, the original keyword candidate is deleted from a keyword candidate set. If the result of this removal is not an empty character string, the original keyword candidate is replaced by a keyword candidate in which an unnecessary head character string is removed.

12. A keyword extracting device of claim 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 or 11 wherein the pattern storage means has at least the keyword component pattern representing a character string capable of being a component of a keyword and an unnecessary end pattern representing an unnecessary end character string,

and further comprising an unnecessary end character string removing means for removing a portion matched with the unnecessary end pattern from a keyword candidate matched with the unnecessary end pattern, by using the pattern processing means for an end character string of each keyword candidate of the keyword candidate set, if this removal results in an empty character string, the original keyword candidate is deleted from a keyword candidate set. If the result of this removal is not an empty character string, the original keyword candidate is replaced by a keyword candidate in which an unnecessary end character string is removed.

13. A keyword extracting device of claim 12 wherein the pattern storage means has at least the keyword component pattern representing a character string capable of being a component of a keyword, the unnecessary end pattern representing an unnecessary end character string and a necessary end pattern representing a necessary end character string, and the unnecessary end character string removing means removes a portion matched with the unnecessary end pattern from a keyword candidate which is not matched with the necessary end pattern but is matched with the unnecessary end pattern, by using the pattern processing means for an end character string of each keyword candidate of the keyword candidate set, if this removal results in an empty character string, the original keyword candidate is deleted from a keyword candidate set. If the result of this removal is not an empty character string, the original keyword candidate is replaced by a keyword candidate in which an unnecessary end character string is removed.

14. A keyword extracting device of claim 8, 9, 10, 11, 12 or 13 wherein a keyword candidate, which is finally removed from a candidate set by applying two of the unnecessary word removing means, the unnecessary head character string removing means and the unnecessary end character string removing means or more in combination or by applying either the unnecessary head character string removing means or the unnecessary end character string removing means twice or more, is added to the unnecessary word pattern as a selection choice of.

15. A keyword extracting device of claim 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13 or 14 having a plurality of pattern storage means corresponding to a type and field of a text, a kind stored pattern and the like, wherein the pattern processing means simultaneously applies the same kind of patterns stored in each pattern storage means when applying various patterns and sets, as a matching result, the longest character string which is matched at the headmost position in the text.

16. A keyword extracting device of claim 15 wherein the pattern processing means simultaneously applies the pattern stored in each pattern storage means and sets, as a matching result, a character string matched with any pattern, thereby stopping application of other patterns, in relation to the unnecessary word pattern and the necessary word pattern.

17. A keyword extracting device of claim 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13 or 14 comprising a plurality of pattern storage means corresponding to a type and field of a text, a kind of stored pattern and the like, and pattern integrating means for expanding selection, concatenation, repetition and the like in the pattern if necessary, and for integrating and outputting, for each kind of pattern, various patterns stored in the pattern storage means while aligning a component of a pattern in relation to a character string length which might be matched.

18. A keyword extracting device of claim 17 wherein the pattern integrating means for provides an upper limit for a character string length which might be matched, and integrates and outputs, for each kind of pattern, various patterns stored in the pattern storage means while removing a component of a pattern which exceeds the upper limit.

19. A keyword extracting device of claim 17 or 18 wherein the pattern integrating means provides an upper limit of the number of expansions for the repetition of patterns to exclude the repetition of the pattern which exceeds the upper limit, thereby integrates and outputs, for each kind of pattern, various patterns stored in the pattern storage means.

20. A keyword extracting device of claim 17,18 or 19 wherein the pattern integrating means outputs a pattern having, as a selection choice, the pattern stored in each pattern storage means in relation to the unnecessary word pattern and the necessary word pattern.

21. A keyword extracting device comprising text data input means for inputting a text,

part-of-speech analyzing means to analyze a part-of-speech of each word in the input text,

keyword candidate set generating means for generating, as a keyword candidate, a word string corresponding to a prescribed sequence of part-of-speeches from a result of the part-of-speech analyzing,

and keyword output means for outputting each keyword candidate of the keyword candidate set as a keyword.

22. A keyword extracting device of claim 21 wherein the keyword candidate set generating means unifies keyword candidates repeated in the keyword candidate set into one keyword candidates.

23. A keyword extracting device of claim 21 or 22 further comprising:

pattern processing means for carrying out matching and replacement of a character string based on pattern in a regular expression or its equivalent,

and pattern storage means for storing a pattern representing a character string to be used for the matching and the replacement in the pattern processing means.

24. A keyword extracting device of claim 23 wherein the pattern storage means has a non-keyword candidate pattern representing a portion which cannot be a candidate of a keyword,

and further comprising non-keyword removing means for searching all character strings which are matched with the non-keyword candidate pattern and are not repeated with each other and for replacing the character string with a special character string which does not correspond to the prescribed sequence of part-of-speeches by using the pattern processing means for the text.

25. A keyword extracting device of claim 23 or 24 wherein the pattern storage means has an extra keyword component pattern representing a keyword component which is hard to generate as a word string corresponding to the prescribed sequence of part-of-speeches,

and further comprising extra keyword component extracting means for extracting, as extra keyword components, all character strings which are matched with the extra keyword component pattern and are not overlapped with each other, recording an extracting position, and converting each character string matched in the text into a special character string which does not correspond to the prescribed sequence of part-of-speeches by using pattern processing means for the text,

and the keyword candidate set generating means generates the keyword candidate set from the word string corresponding to the prescribed sequence of part-of-speeches and the extra keyword component.

26. A keyword extracting device of claim 23, 24 or 25 wherein the pattern storage means has an unnecessary word pattern representing an unnecessary word,

and comprising unnecessary word removing means for removing a character string matched with the unnecessary word pattern from the keyword candidate set by using pattern processing means for a whole character string of each keyword candidate of the keyword candidate set.

27. A keyword extracting device of claim 26 wherein the pattern storage means has a necessary word pattern representing a necessary word and the unnecessary word pattern representing an unnecessary word, and the unnecessary word removing means for removing a character string which is not matched with the necessary word pattern but is matched with the unnecessary word pattern from the keyword candidate set by using pattern processing means for a whole character string of each keyword candidate of the keyword candidate set.

28. A keyword extracting device of claim 23, 24, 25, 26 or 27 wherein the pattern storage means has an unnecessary head pattern representing an unnecessary head character string,

and further comprising unnecessary head character string removing means for removing a portion matched with the unnecessary head pattern from a keyword candidate matched with the unnecessary head pattern, by using the pattern processing means for a head character string of each keyword candidate of the keyword candidate set, if this removal results in an empty character string, the original keyword candidate is deleted from a keyword candidate set. If the result of this removal is not an empty character string, the original keyword candidate is replaced by a keyword candidate in which an unnecessary head character string is removed.

29. A keyword extracting device of claim 28 wherein the pattern storage means has the unnecessary head pattern representing an unnecessary head character string and a necessary head pattern representing a necessary head character string, and the unnecessary head character string removing means removes a portion matched with the unnecessary head pattern from a keyword candidate which is not matched with the necessary head pattern but is matched with the unnecessary head pattern, by using the pattern processing means for a head character string of each keyword candidate of the keyword candidate set, if this removal results in an empty character string, the original keyword candidate is deleted from a keyword candidate set. If the result of this removal is not an empty character string, the original keyword candidate is replaced by a keyword candidate in which an unnecessary head character string is removed.

30. A keyword extracting device of claim 23, 24, 25, 26, 27, 28 or 29 wherein the pattern storage means has an unnecessary end pattern representing an unnecessary end character string,

and further comprising unnecessary end character string removing means for removing a portion matched with the unnecessary end pattern from a keyword candidate matched with the unnecessary end pattern, by using pattern processing means for an end character string of each keyword candidate of the keyword candidate set, if this removal results in an empty character string, the original keyword candidate is deleted from a keyword candidate set. If the result of this removal is not an empty character string, the original keyword candidate is replaced by a keyword candidate in which an unnecessary end character string is removed.

31. A keyword extracting device of claim 30 wherein the pattern storage means has the unnecessary end pattern representing an unnecessary end character string and a necessary end pattern representing a necessary end character string, and the unnecessary end character string removing means removes a portion matched with the unnecessary end pattern from a keyword candidate which is not matched with the necessary end pattern but is matched with the unnecessary end pattern, by using pattern processing means for an end character string of each keyword candidate of the keyword candidate set, if this removal results in an empty character string, the original keyword candidate is deleted from a keyword candidate set. If the result of this removal is not an empty character string, the original keyword candidate is replaced by a keyword candidate in which an unnecessary end character string is removed.

32. A keyword extracting device of claim 26, 27, 28, 29, 30 or 31 wherein a keyword candidate, which is finally removed from a candidate set by applying two of the unnecessary word removing means, the unnecessary head character string removing means and the unnecessary end character string removing means or more in combination or by applying either the unnecessary head character string removing means or the unnecessary end character string removing means twice or more, is added to the unnecessary word pattern as a selection choice of.

33. A keyword extracting device of claim 23, 24, 25, 26, 27, 28, 29, 30, 31 or 32 having a plurality of pattern storage means corresponding to a type and field of a text, a kind stored pattern and the like, wherein the pattern processing means simultaneously applies the same kind of patterns stored in each pattern storage means when applying various patterns and sets, as a matching result, the longest character string which is matched at the headmost position in the text.

34. A keyword extracting device of claim 33 wherein the pattern processing means simultaneously applies the pattern stored in each pattern storage means and sets, as a matching result, a character string matched with any pattern, thereby stopping application of other patterns, in relation to the unnecessary word pattern and the necessary word pattern.

35. A keyword extracting device of claim 23, 24, 25, 26, 27, 28, 29, 30, 31, 32 or 33 comprising a plurality of pattern storage means corresponding to a type and field of a text, a kind of stored pattern and the like, and pattern integrating means for expanding selection, concatenation, repetition and the like in the pattern if necessary, and for integrating and outputting, for each kind of pattern, various patterns stored in the pattern storage means while aligning a component of a pattern in relation to a character string length which might be matched.

36. A keyword extracting device of claim 35 wherein the pattern integrating means for provides an upper limit for a character string length which might be matched, and integrates and outputs, for each kind of pattern, various patterns stored in the pattern storage means while removing a component of a pattern which exceeds the upper limit.

37. A keyword extracting device of claim 35, 36 wherein the pattern integrating means provides an upper limit of the number of expansions for the repetition of patterns to exclude the repetition of the pattern which exceeds the upper limit, thereby integrates and outputs, for each kind of pattern, various patterns stored in the pattern storage means.

38. A keyword extracting device of claim 35, 36 or 37 wherein the pattern integrating means outputs a pattern having, as a selection choice, the pattern stored in each pattern storage means in relation to the unnecessary word pattern and the necessary word pattern.