Dictionary creation device and dictionary creation method

• US 20060242191A1
• Filed: 06/21/2006
• Published: 10/26/2006
• Est. Priority Date: 12/26/2003
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Naito, Eiichi, Ozawa, Jun, Kutsumi, Hiroshi, Yoshida, Hideyuki
• Granted Patent: US 7,840,565 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: H04N 21/482 End-user interface for program selection broadcast systems using EPGs H04H60/72

Abstract

There is provided a dictionary creation device and dictionary creation method which optimally creates and updates a dictionary for classifying, searching, or extracting text information in accordance with a changes in content of text information groups. The dictionary creation device includes: a keyword extraction unit that extracts a keyword from inputted text information; a keyword statistics unit that finds statistics regarding an appearance of the keyword; a keyword assessment value calculation unit that calculates an assessment value of the extracted keyword based on the statistics regarding the appearance of the keyword; a determination unit that determines whether or not to register or delete the keyword based on the calculated assessment value; a dictionary registration and deletion unit which registers or deletes is the keyword in or from a dictionary database based on a result of the determination performed by the determination unit; and the dictionary database.

Claims

1. A dictionary creation device that creates a dictionary used for searching, classifying, or filtering information written as text, said device comprising:

a keyword extraction unit operable to extract a keyword from a text information group made up of one or more pieces of text information;

a keyword statistics unit operable to find statistics regarding an appearance of the keyword within the text information group;

a keyword assessment value calculation unit operable to calculate an assessment value for the keyword based on the statistics;

a keyword storage unit operable to store a pair made up of the keyword and the assessment value of that keyword;

a determination unit operable to determine whether or not to register the keyword in the dictionary, or whether or not to delete the keyword from the dictionary, based on a degree of change between the assessment value newly calculated by said keyword assessment value calculation unit and the assessment value stored in said keyword storage unit; and

a dictionary registration and deletion unit operable to register or delete the keyword in the dictionary based on a result of the determination.

2. The dictionary creation device according to claim 1, further comprising

a second keyword extraction unit operable to extract a keyword from a second text information group made up of one or more pieces of text information,

wherein said keyword statistics unit is further operable to find statistics regarding an appearance of the keyword within the second text information group, and

said keyword assessment value calculation unit is operable to calculate the assessment values of the keywords extracted by said keyword extraction unit and said second keyword extraction unit based on the respective statistics.

3. The dictionary creation device according to claim 1,

wherein the assessment value calculated by said keyword assessment value calculation unit is an inverse document frequency (idf) value of the keyword based on a total number of pieces of text information in the text information group and a number of pieces of text information that include the keyword, and

said determination unit is operable to determine to register the keyword in the dictionary in the case where the keyword is not yet registered in the dictionary and a degree of change in the idf value is greater than or equal to a predetermined threshold value.

4. The dictionary creation device according to claim 3,

wherein said determination unit is operable to determine to delete the keyword from the dictionary in the case where the keyword is already registered in the dictionary and a degree of change in the idf value is less than a predetermined threshold value.

5. The dictionary creation device according to claim 1,

wherein the assessment value calculated by said keyword assessment value calculation unit is an inverse document frequency (idf) value of the keyword based on a total number of pieces of text information in the text information group and a number of pieces of text information that include the keyword;

the dictionary is an unnecessary word dictionary; and

said determination unit is operable to determine to register the keyword in the unnecessary word dictionary in the case where the keyword is not yet registered in the unnecessary word dictionary and a degree of change in the idf value is less than a predetermined threshold value.

6. The dictionary creation device according to claim 5,

wherein said determination unit is operable to determine to delete the keyword from the unnecessary word dictionary in the case where the keyword is already registered in the unnecessary word dictionary and a degree of change in the idf value is greater than or equal to a predetermined threshold value.

7. The dictionary creation device according to claim 1,

wherein the assessment value calculated by said keyword assessment value calculation unit is an appearance frequency at which the keyword appears in the text information group, and

said determination unit is operable to determine to register the keyword in the dictionary in the case where the keyword is not yet registered in the dictionary and a degree of change in the appearance frequency is less than a predetermined threshold value.

8. The dictionary creation device according to claim 7,

wherein said determination unit is operable to determine to delete the keyword from the dictionary in the case where the keyword is already registered in the dictionary and a degree of change in the appearance frequency is greater than or equal to a predetermined threshold value.

9. The dictionary creation device according to claim 1,

wherein the assessment value calculated by said keyword assessment value calculation unit is an appearance frequency at which the keyword appears in the text information group;

the dictionary is an unnecessary word dictionary; and

said determination unit is operable to determine to register the keyword in the unnecessary word dictionary in the case where the keyword is not yet registered in the unnecessary word dictionary and a degree of change in the appearance frequency is greater than or equal to a predetermined threshold value.

10. The dictionary creation device according to claim 9,

wherein said determination unit is operable to determine to delete the keyword from the unnecessary word dictionary in the case where the keyword is already registered in the unnecessary word dictionary and a degree of change in the appearance frequency is less than a predetermined threshold value.

11. The dictionary creation device according to claim 1,

wherein the text information is Electric Program Guide (EPG) data;

the dictionary is created based on a type of broadcast wave;

said keyword extraction unit is operable to extract a keyword from the text information group based on the type of the broadcast wave; and

said dictionary registration and deletion unit is operable to register or delete the keyword in the dictionary based on the type of the broadcast wave.

12. The dictionary creation device according to claim 1,

wherein the text information is Electric Program Guide (EPG) data;

the dictionary is created based on a time period of a broadcast;

said keyword extraction unit is operable to extract a keyword from the text information group based on the time period of the broadcast; and

said dictionary registration and deletion unit is operable to register or delete the keyword in the dictionary based on the time period of the broadcast.

13. A dictionary creation method for creating a dictionary used for searching, classifying, or filtering information written as text, said method comprising:

extracting a keyword from a text information group made up of one or more pieces of text information;

finding statistics regarding an appearance of the keyword within the text information group;

calculating an assessment value for the keyword based on the statistics;

storing, in a keyword storage unit, a pair made up of the keyword and the assessment value of that keyword;

determining whether or not to register the keyword in the dictionary, or whether or not to delete the keyword from the dictionary, based on a degree of change between the assessment value newly calculated in said calculating and the assessment value stored in said keyword storage unit; and

registering or deleting the keyword in the dictionary based on a result of the determination.

14. A program for creating a dictionary used for searching, classifying, or filtering information written as text, said program causing a computer to execute the steps of:

extracting a keyword from a text information group made up of one or more pieces of text information;

finding statistics regarding an appearance of the keyword within the text information group;

calculating an assessment value for the keyword based on the statistics;

storing, in a keyword storage unit, a pair made up of the keyword and the assessment value of that keyword;

determining whether or not to register the keyword in the dictionary, or whether or not to delete the keyword from the dictionary, based on a degree of change between the assessment value newly calculated in said calculating and the assessment value stored in said keyword storage unit; and

registering or deleting the keyword in the dictionary based on a result of the determination.