DICTIONARY GENERATION DEVICE, DICTIONARY GENERATION METHOD, DICTIONARY GENERATION PROGRAM AND COMPUTER-READABLE RECORDING MEDIUM STORING SAME PROGRAM

• US 20150012264A1
• Filed: 02/15/2013
• Published: 01/08/2015
• Est. Priority Date: 02/15/2012
• Status: Active Grant

• Current Assignee: Rakuten Group, Inc.
• Original Assignee: Rakuten, Inc. (Rakuten Group, Inc.)
• Inventors: Murakami, Koji
• Granted Patent: US 9,430,793 B2
• Time in Patent Office: Days
• US Class Current: 704/10
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/268 Morphological analysis
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/0623 Item investigation

Abstract

A dictionary generation device according to one embodiment includes a determination unit configured to (A) refer to an item database that stores a plurality of records containing an item name/item description including a noun sequence, an item category, and a shop selling the item as fields and determine whether the noun sequence included in the item name/item description of each record is set corresponding to the item category, (B) count the number of selling shops in a record containing the noun sequence for each item category and calculate a shop intensity of each noun sequence based on the counted number of selling shops, (C) determine whether one item category uniquely derived from the noun sequence exists based on the shop intensity for each item category, and (D) determine the noun sequence as a definitive category word when the one item category exists.

Claims

1. A dictionary generation device comprising:

a determination unit configured to (A) refer to an item database that stores a plurality of records containing an item name and/or item description including a noun, a noun phrase or a noun sequence being a sequence of nouns or noun phrases, an item category, and a shop selling the item as fields and determine whether the noun sequence included in the item name and/or item description of each record is set corresponding to the item category, (B) count the number of selling shops in a record containing the noun sequence set corresponding to the item category for each item category and calculate a shop intensity of each noun sequence set corresponding to the item category based on the counted number of selling shops, (C) determine whether one item category uniquely derived from the noun sequence exists based on the shop intensity for each item category, and (D) determine the noun sequence as a definitive category word when the one item category exists; and

a registration unit configured to store dictionary information where the definitive category word and the uniquely derived item category are associated with each other into a dictionary database.

2. The dictionary generation device according to claim 1, wherein

the determination unit calculates the number of appearances of the noun sequence for each item category, calculates a noun sequence intensity for each item category based on the number of appearances, and determines whether one item category uniquely derived from the noun sequence exists based on the shop intensity and the noun sequence intensity for each item category.

3. The dictionary generation device according to claim 1, wherein

the determination unit calculates the number of items corresponding to each item category, calculates an item intensity for each item category based on the number of items, and determines whether one item category uniquely derived from the noun sequence exists based on the shop intensity and the item intensity for each item category.

4. The dictionary generation device according to claim 1, wherein

the determination unit calculates a total number of shops having registered item information where the item name includes the noun sequence, and when the total number of shops is equal to or less than a specified threshold, does not determine the noun sequence as the definitive category word.

5. The dictionary generation device according to claim 1, wherein

the determination unit refers to an unnecessary words dictionary that stores unnecessary words, and when the noun sequence coincides with the unnecessary word, does not determine the noun sequence as the definitive category word.

6. The dictionary generation device according to claim 1, wherein

the determination unit specifies an item category with the highest shop intensity as a major category and specifies an item category with a lower shop intensity than the major category as a minor category, calculates an overlap degree of words between item names of the minor category and item names of the major category, and when the overlap degree is equal to or higher than a specified threshold, determines that the major category is the item category uniquely derived from the noun sequence.

7. The dictionary generation device according to claim 6, wherein

when the overlap degree is equal to or higher than the specified threshold in N-th hierarchical level of the major category and the minor category, the determination unit determines that the N-th hierarchical level of the major category is uniquely derived from the noun sequence, and then repeats the specification of the major category and the minor category, the calculation of the overlap degree and the determination based on the overlap degree in (N+1)th hierarchical level of the major category, where N is 1 or more.

8. The dictionary generation device according to claim 1, further comprising:

an update unit configured to update an item category in the item information where an item name includes the definitive category word indicated by the dictionary information to an item category indicated by the dictionary information, wherein

after the item information is updated by the update unit, processing by the determination unit, the registration unit and the update unit is performed repeatedly, and

the determination unit determines whether one item category uniquely derived from the noun sequence exists based on the shop intensity by setting a threshold for the shop intensity to be lower than a value used in the previous determination.

9. The dictionary generation device according to claim 1, wherein

the determination unit extracts the noun sequence from the item name by morphological analysis.

10. The dictionary generation device according to claim 1, wherein

the determination unit extracts the noun sequence from the item name by reference to a rule dictionary that stores a description rule of item names indicating an extraction range of the noun sequence.

11. The dictionary generation device according to claim 1, wherein

the determination unit extracts words different from unnecessary words from the item name by reference to an unnecessary words dictionary that stores unnecessary words, and extracts the noun sequence from the extracted words.

12. The dictionary generation device according to claim 1, wherein

the registration unit stores recommendation information where the noun sequence and an item category different from the uniquely derived item category among the item categories are associated with each other into another dictionary database.

13. The dictionary generation device according to claim 1, wherein

the determination unit determines whether one item category uniquely derived from a pair of the noun sequence and a shop or a shop category exists based on the shop intensity for each item category, and determines the noun sequence as the definitive category word when the one item category exists, and

the registration unit stores dictionary information where the definitive category word, the shop or the shop category and the uniquely derived item category are associated with one another into the dictionary database.

14. The dictionary generation device according to claim 1, wherein

the determination unit extracts the noun sequence from the item name by reference to a notation dictionary that stores information about notational variants.

15. The dictionary generation device according to claim 1, wherein

the registration unit extracts a word corresponding to the definitive category word by reference to a notation dictionary that stores information about notational variants, and stores dictionary information where the definitive category word, the corresponding word and the uniquely derived item category are associated with one another into the dictionary database.

16. The dictionary generation device according to claim 1, wherein

when there is only one item category where the shop intensity is higher than a first threshold, the determination unit determines that one item category uniquely derived from the noun sequence exists and determines the noun sequence as a Type 1 definitive category word of the item category,

when there are a plurality of item categories where the shop intensity is higher than the first threshold, the determination unit determines the noun sequence as a Type 2 definitive category word with less definitiveness of the item category than the Type 1 definitive category word or a reference word with less definitiveness of the item category than the Type 2 definitive category word for each of the plurality of item categories, and

the registration unit stores dictionary information where the Type 1 definitive category word, the Type 2 definitive category word or the reference word and the item category are associated with each other into the dictionary database.

17. The dictionary generation device according to claim 16, wherein

when the number of item categories where the shop intensity is higher than the first threshold is not 1, the determination unit determines whether the number of registration shops of the noun sequence in remaining categories excluding the item category is larger than a second threshold, and

when the number of registration shops is larger than the second threshold, the determination unit determines the noun sequence as an unnecessary word, and

the registration unit stores the unnecessary word into a specified database.

18. A dictionary generation method performed by a dictionary generation device, the method comprising:

a determination step of (A) referring to an item database that stores a plurality of records containing an item name and/or item description including a noun, a noun phrase or a noun sequence being a sequence of nouns or noun phrases, an item category, and a shop selling the item as fields and determining whether the noun sequence included in the item name and/or item description of each record is set corresponding to the item category, (B) counting the number of selling shops in a record containing the noun sequence set corresponding to the item category for each item category and calculating a shop intensity of each noun sequence set corresponding to the item category based on the counted number of selling shops, (C) determining whether one item category uniquely derived from the noun sequence exists based on the shop intensity for each item category, and (D) determining the noun sequence as a definitive category word when the one item category exists; and

a registration step of storing dictionary information where the definitive category word and the uniquely derived item category are associated with each other into a dictionary database.

19. (canceled)

20. A non-transitory computer-readable recording medium storing a dictionary generation program causing a computer to implement:

a determination unit configured to (A) refer to an item database that stores a plurality of records containing an item name and/or item description including a noun, a noun phrase or a noun sequence being a sequence of nouns or noun phrases, an item category, and a shop selling the item as fields and determine whether the noun sequence included in the item name and/or item description of each record is set corresponding to the item category, (B) count the number of selling shops in a record containing the noun sequence set corresponding to the item category for each item category and calculate a shop intensity of each noun sequence set corresponding to the item category based on the counted number of selling shops, (C) determine whether one item category uniquely derived from the noun sequence exists based on the shop intensity for each item category, and (D) determine the noun sequence as a definitive category word when the one item category exists; and

a registration unit configured to store dictionary information where the definitive category word and the uniquely derived item category are associated with each other into a dictionary database.