Dictionary creation device and dictionary creation method

• US 7,840,565 B2
• Filed: 06/21/2006
• Issued: 11/23/2010
• Est. Priority Date: 12/26/2003
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Kutsumi, Hiroshi, Ozawa, Jun, Naito, Eiichi, Yoshida, Hideyuki
• Primary Examiner(s): Jalil, Neveen Abel
• Assistant Examiner(s): Syed, Farhan M
• Application Number:
  US11/471,538
• Publication Number:
  US 20060242191A1
• Time in Patent Office: 1,616 Days
• Field of Search: None
• US Class Current: 707/740
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: H04N 21/482 End-user interface for program selection broadcast systems using EPGs H04H60/72

Abstract

A dictionary creation device and dictionary creation method are provided which optimally create and update a dictionary for classifying, searching, or extracting text information in accordance with a changes in content of text information groups. The dictionary creation device includes a keyword extraction unit that extracts a keyword from inputted text information and a keyword statistics unit that finds statistics regarding an appearance of the keyword. The dictionary creation device further includes a keyword assessment value calculation unit that calculates an assessment value of the extracted keyword based on the statistics regarding the appearance of the keyword, a determination unit that determines whether or not to register or delete the keyword based on the calculated assessment value, a dictionary registration and deletion unit which registers or deletes the keyword in or from a dictionary database based on a result of the determination performed by the determination unit, and the dictionary database.

Claims

1. A dictionary creation device that creates a dictionary used for searching, classifying, or filtering information written as text, said device comprising:

a keyword extraction unit configured to extract a keyword from a text information group made up of one or more pieces of text information;

a keyword statistics unit configured to find statistics regarding an appearance of the keyword within the text information group;

a keyword assessment value calculation unit configured to calculate, using a processor, an assessment value for the keyword based on the statistics;

a keyword storage unit configured to store a pair made up of the keyword and the assessment value of that keyword, said keyword storage unit being a memory unit;

a determination unit configured to determine whether or not to register the keyword in the dictionary, or whether or not to delete the keyword from the dictionary, based on a degree of change between the assessment value newly calculated by said keyword assessment value calculation unit and the assessment value stored in said keyword storage unit; and

a dictionary registration and deletion unit configured to register or delete the keyword in the dictionary based on a result of the determination,

wherein the assessment value calculated by said keyword assessment value calculation unit is an appearance frequency at which the keyword appears in the text information group,

wherein said determination unit is configured to determine to delete the keyword from the dictionary in the case where the keyword is already registered in the dictionary and a degree of change in the appearance frequency is greater than or equal to a predetermined threshold value, the degree of change in the appearance frequency indicating a difference between the appearance frequency and a previously calculated appearance frequency, and

wherein the same keyword is used for calculating the appearance frequency and the previously calculated appearance frequency.

2. The dictionary creation device according to claim 1, further comprising

a second keyword extraction unit configured to extract a keyword from a second text information group made up of one or more pieces of text information,

wherein said keyword statistics unit is further configured to find statistics regarding an appearance of the keyword within the second text information group, and

said keyword assessment value calculation unit is configured to calculate the assessment values of the keywords extracted by said keyword extraction unit and said second keyword extraction unit based on the respective statistics.

3. The dictionary creation device according to claim 1,

wherein the assessment value calculated by said keyword assessment value calculation unit is an inverse document frequency (idf) value of the keyword based on a total number of pieces of text information in the text information group and a number of pieces of text information that include the keyword, and

said determination unit is configured to determine to register the keyword in the dictionary in the case where the keyword is not yet registered in the dictionary and a degree of change in the idf value is greater than or equal to a predetermined threshold value.

4. The dictionary creation device according to claim 3,

wherein said determination unit is configured to determine to delete the keyword from the dictionary in the case where the keyword is already registered in the dictionary and a degree of change in the idf value is less than a predetermined threshold value.

5. The dictionary creation device according to claim 1,

wherein the assessment value calculated by said keyword assessment value calculation unit is an inverse document frequency (idf) value of the keyword based on a total number of pieces of text information in the text information group and a number of pieces of text information that include the keyword;

the dictionary is an unnecessary word dictionary; and

said determination unit is configured to determine to register the keyword in the unnecessary word dictionary in the case where the keyword is not yet registered in the unnecessary word dictionary and a degree of change in the idf value is less than a predetermined threshold value.

6. The dictionary creation device according to claim 5,

wherein said determination unit is configured to determine to delete the keyword from the unnecessary word dictionary in the case where the keyword is already registered in the unnecessary word dictionary and a degree of change in the idf value is greater than or equal to a predetermined threshold value.

7. The dictionary creation device according to claim 1,

wherein said determination unit is configured to determine to register the keyword in the dictionary in the case where the keyword is not yet registered in the dictionary and a degree of change in the appearance frequency is less than a predetermined threshold value.

8. The dictionary creation device according to claim 1,

wherein the dictionary is an unnecessary word dictionary;

and

said determination unit is configured to determine to register the keyword in the unnecessary word dictionary in the case where the keyword is not yet registered in the unnecessary word dictionary and a degree of change in the appearance frequency is greater than or equal to a predetermined threshold value.

9. The dictionary creation device according to claim 8,

wherein said determination unit is configured to determine to delete the keyword from the unnecessary word dictionary in the case where the keyword is already registered in the unnecessary word dictionary and a degree of change in the appearance frequency is less than a predetermined threshold value.

10. The dictionary creation device according to claim 1,

wherein the text information is Electric Program Guide (EPG) data;

the dictionary is created based on a type of broadcast wave;

said keyword extraction unit is configured to extract a keyword from the text information group based on the type of the broadcast wave; and

said dictionary registration and deletion unit is configured to register or delete the keyword in the dictionary based on the type of the broadcast wave.

11. The dictionary creation device according to claim 1,

wherein the text information is Electric Program Guide (EPG) data;

the dictionary is created based on a time period of a broadcast;

said keyword extraction unit is configured to extract a keyword from the text information group based on the time period of the broadcast; and

said dictionary registration and deletion unit is configured to register or delete the keyword in the dictionary based on the time period of the broadcast.

12. A dictionary creation method for creating a dictionary used for searching, classifying, or filtering information written as text, said method comprising:

extracting a keyword from a text information group made up of one or more pieces of text information;

finding statistics regarding an appearance of the keyword within the text information group;

calculating, using a processor, an assessment value for the keyword based on the statistics;

storing, in a keyword storage unit, a pair made up of the keyword and the assessment value of that keyword, said keyword storage unit being a memory unit;

determining whether or not to register the keyword in the dictionary, or whether or not to delete the keyword from the dictionary, based on a degree of change between the assessment value newly calculated in said calculating and the assessment value stored in said keyword storage unit; and

registering or deleting the keyword in the dictionary based on a result of the determination,

wherein the assessment value calculated in said calculating is an appearance frequency at which the keyword appears in the text information group,

wherein said determining comprises determining to delete the keyword from the dictionary in the case where the keyword is already registered in the dictionary and a degree of change in the appearance frequency is greater than or equal to a predetermined threshold value, the degree of change in the appearance frequency indicating a difference between the appearance frequency and a previously calculated appearance frequency, and

wherein the same keyword is used for calculating the appearance frequency and the previously calculated appearance frequency.

13. A program embodied on a non-transitory computer-readable storage medium for creating a dictionary used for searching, classifying, or filtering information written as text, said program causing a computer to execute the steps of:

extracting a keyword from a text information group made up of one or more pieces of text information;

finding statistics regarding an appearance of the keyword within the text information group;

calculating, using a processor, an assessment value for the keyword based on the statistics;

storing, in a keyword storage unit, a pair made up of the keyword and the assessment value of that keyword, said keyword storage unit being a memory unit;

determining whether or not to register the keyword in the dictionary, or whether or not to delete the keyword from the dictionary, based on a degree of change between the assessment value newly calculated in said calculating and the assessment value stored in said keyword storage unit; and

registering or deleting the keyword in the dictionary based on a result of the determination,

wherein the assessment value calculated in said calculating is an appearance frequency at which the keyword appears in the text information group,

wherein said determining comprises determining to delete the keyword from the dictionary in the case where the keyword is already registered in the dictionary and a degree of change in the appearance frequency is greater than or equal to a predetermined threshold value, the degree of change in the appearance frequency indicating a difference between the appearance frequency and a previously calculated appearance frequency, and

wherein the same keyword is used for calculating the appearance frequency and the previously calculated appearance frequency.