Method and apparatus for updating dictionary

• US 20060100856A1
• Filed: 11/09/2005
• Published: 05/11/2006
• Est. Priority Date: 11/09/2004
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Cho, Jeongmi, Kang, Inho
• Granted Patent: US 8,311,807 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/49 using very large corpora, e.g. the web

Abstract

A method and apparatus for updating a dictionary, by which documents of interest of a user are extracted through a network and a dictionary is updated with individual names and coined words extracted from the web documents, are provided. The dictionary updating method includes: extracting a document of interest of a user in each predetermined extraction period from a predetermined server connected to a dictionary updating system through a network, and extracting candidate unregistered words existing in the extracted document according to predetermined unregistered word extraction rules; based on information on appearance frequencies of the candidate unregistered words retrieved from the document, extracting unregistered words among the candidate unregistered words and extracting candidate semantic classes of the unregistered word; with usage examples of the unregistered word obtained through a searching unit, according to a predetermined unregistered word verification method, verifying the unregistered word and determining the semantic class of the verified unregistered word; and updating the dictionary updating system with the verified unregistered word and the semantic class of the verified unregistered word.

Claims

1. A method for updating a dictionary in a dictionary updating system with an unregistered word not registered in the dictionary, the method comprising:

extracting a document of interest of a user in each predetermined extraction period from a predetermined server connected to the dictionary updating system through a network, and extracting candidate unregistered words existing in the extracted document according to predetermined unregistered word extraction rules;

extracting unregistered words among the candidate unregistered words and extracting candidate semantic classes of the unregistered word based on information on appearance frequencies of the candidate unregistered words retrieved from the document;

verifying the unregistered word according to a predetermined unregistered word verification method and determining the semantic class of the verified unregistered word with usage examples of the unregistered word obtained through a searching unit; and

updating the dictionary updating system with the verified unregistered word and the semantic class of the verified unregistered word.

2. The method of claim 1, wherein the candidate unregistered word is an individual name formed with a word or a set of words not registered in the dictionary.

3. The method of claim 2, wherein the extracting of unregistered words among the candidate unregistered words and extracting of candidate semantic classes of the unregistered word comprises:

calculating the appearance frequency of the candidate unregistered word in the document extracted during the extraction period;

calculating the changed appearance frequency value of the candidate unregistered word in each extraction period;

selecting a candidate unregistered word having an appearance frequency equal to or greater than a first threshold and a changed appearance frequency value equal to or greater than a second threshold, as an unregistered word; and

extracting a candidate semantic class of the unregistered word from the document.

4. The method of claim 3, further comprising:

determining the weight of the extracted candidate unregistered word with respect to the type of the document, wherein the appearance frequency is weighted by the weight.

5. The method of claim 4, wherein in determining the weight, a first weight, a second weight, and a third weight are allocated to the candidate unregistered words retrieved from a table, a new article, and an ordinary document of the documents, respectively, and the first weight is greater than the second weight, and the second weight is greater than the third weight.

6. The method of claim 2, wherein the verifying of the unregistered word and the determining of the semantic class of the verified unregistered word comprise:

generating a search word for the selected unregistered word and performing search through a searching unit;

extracting a usage example of the unregistered word from the search result for the search word;

verifying the unregistered word with the usage example according to a predetermined unregistered word verification method; and

determining the semantic class of the unregistered word among candidate semantic classes of the unregistered word according to the usage example.

7. The method of claim 6, wherein according to the unregistered word verification method, an individual name having the ratio of the frequency value of each word forming the individual name to the frequency value of the individual name in the usage example of the unregistered word, exceeding a third threshold is verified as an unregistered word.

8. The method of claim 6, wherein according to the unregistered word verification method, it is verified whether the unregistered word is expressed by any one of parentheses, quotation marks, brackets, a bold letter attribute, and an underline.

9. The method of claim 2, further comprising:

inputting a field of interest of a user and extracting a document from a web address corresponding to the field of interest of the user.

10. The method of claim 2, wherein the dictionary updating system comprises:

a first dictionary registering an unregistered word; and

a second dictionary re-registering a registered word registered in the first dictionary, and the method further comprises;

among registered words registered in the first dictionary, re-registering a registered word in the second dictionary, based on the user'"'"'s usage frequency of the registered word and appearance frequency information in a document of the registered word.

11. The method of claim 10, wherein the re-registering of the registered word comprises:

calculating the user'"'"'s usage frequency of the registered word among registered words registered in the first dictionary;

calculating the appearance frequency in a document of the registered word among registered words registered in the first dictionary, and the changed appearance frequency value of the registered word in each extraction period;

retrieving a registered word satisfying at least one of the user'"'"'s usage frequency equal to or greater than a fourth threshold, the appearance frequency equal to or greater than a fifth threshold, and the changed appearance frequency value equal to or greater than a sixth threshold; and

determining whether the retrieved registered word is registrable in the second dictionary and re-registering the retrieved registered word in the second dictionary.

12. The method of claim 11, wherein the re-registering of the retrieved registered word comprises:

determining whether there is a space for registering the retrieved registered word, in the second dictionary;

based on the determination result, deleting a registered word having a lowest priority based on the user'"'"'s usage frequency, the appearance frequency, and the changed appearance frequency value of the registered word, among registered words registered in the second dictionary; and

re-registering the retrieved registered word in the second dictionary.

13. The method of claim 10, further comprising:

displaying information on the registered word re-registered in the second dictionary.

14. The method of claim 2, wherein the unregistered word is retrieved according to a predetermined natural language interpretation method, and the individual name is retrieved according to a regular expression method.

15. The method of claim 1, wherein the document is a web document.

16. The method of claim 1, wherein the server is web server.

17. A computer readable recording medium having embodied thereon a computer program for the method of claim 1.

18. A method for updating a dictionary of a voice recognition system providing a service corresponding to a voice command of a user, with an unregistered word not registered in the dictionary, the method comprising:

extracting a document of interest of a user in each predetermined extraction period from a predetermined server connected to the voice recognition system, and retrieving candidate unregistered words existing in the extracted document according to predetermined unregistered word extraction rules;

extracting unregistered words among the candidate unregistered words and extracting candidate semantic classes of the unregistered word based on information on appearance frequencies of the candidate unregistered words retrieved from the document; and

verifying the unregistered word according to a predetermined unregistered word verification method and determining the semantic class of the verified unregistered word with usage examples of the unregistered word obtained through a searching unit;

updating the voice recognition system with the unregistered word and the semantic class of the unregistered word.

19. The method of claim 18, further comprises:

re-registering the voice recognition system based on the user'"'"'s usage frequency of the registered words and the appearance frequency information in a document of the registered word; and

modifying a voice recognition model and a natural language processing model.

20. The method of claim 18, wherein the document is a web document.

21. An apparatus for updating a dictionary comprising:

a document extraction unit to access a server through a network and extracting a document of interest of a user in each predetermined extraction period;

an unregistered word extraction unit to extract candidate unregistered words existing in the extracted document according to predetermined unregistered word extraction rules, and based on appearance frequency information of the candidate unregistered words in the document, to extract unregistered words among the candidate unregistered words;

an unregistered word verification unit to verify the unregistered words with usage examples of the unregistered words extracted through the server, and to determine the semantic classes of the verified unregistered words;

a first memory unit to store the unregistered words and the semantic classes of the unregistered words; and

a registration unit to register the unregistered words and the semantic classes of the unregistered words in a predetermined location of the memory unit.

22. The apparatus of claim 21, wherein the candidate unregistered word is an individual name formed with a word or a set of words not registered in the dictionary.

23. The apparatus of claim 22, wherein the document extraction unit further comprises:

a second memory unit to store addresses corresponding to at least one or more fields of interest of the user;

a selection unit to select an address corresponding to a field of interest of the user;

a network connection unit to connect the dictionary updating apparatus to the selected address of a network; and

an extraction unit to extract a predetermined document from the selected address.

24. The apparatus of claim 22, wherein the unregistered word extraction unit comprises:

a candidate unregistered word extraction unit to extract a candidate unregistered word existed in the extracted document according to a predetermined unregistered word extraction rule;

an unregistered word selection unit to select an unregistered word among candidate unregistered words, based on the appearance frequency information in the document of the candidate unregistered word; and

a semantic class extraction unit to extract candidate semantic classes of the selected unregistered word in the document.

25. The apparatus of claim 24, wherein the unregistered word selection unit further comprises:

an appearance frequency calculation unit to calculate the appearance frequency of the candidate unregistered word in the document extracted during the extraction period;

a changed appearance frequency value calculation unit to calculate the changed appearance frequency value of the candidate unregistered word in each extraction period;

a selection unit to select a candidate unregistered word having an appearance frequency equal to or greater than a first threshold and a changed appearance frequency value equal to or greater than a second threshold, as an unregistered word.

26. The apparatus of claim 25, wherein the unregistered word extraction unit further comprises:

a weight calculation unit to determine the weight of the extracted candidate unregistered word with respect to the type of the document, and the appearance frequency is weighted by the weight.

27. The apparatus of claim 26, wherein in determining the weight, a first weight, a second weight, and a third weight are allocated to the candidate unregistered words retrieved from a table, a new article, and an ordinary document of the documents, respectively, and the first weight is greater than the second weight, and the second weight is greater than the third weight.

28. The apparatus of claim 22, wherein the unregistered word verification unit comprises:

a searching unit to perform search for the extracted unregistered word;

a usage example extraction unit to extract a usage example of the unregistered word from the search result;

a verification unit to verify the unregistered word with the extracted usage example according to a predetermined unregistered word verification method; and

a semantic information determination unit to determine the semantic class of the unregistered word according to the usage example.

29. The apparatus of claim 28, wherein according to the unregistered word verification method, an individual name having the ratio of the frequency value of each word forming the individual name to the frequency value of the individual name in the usage example of the unregistered word, exceeding a third threshold is verified as an unregistered word.

30. The apparatus of claim 28, wherein according to the unregistered word verification method, it is verified whether the unregistered word is expressed by any one of parentheses, quotation marks, brackets, a bold letter attribute, and an underline.

31. The apparatus of claim 22, further comprising:

an user interface unit inputting a field of interest of a user or an address corresponding to the field of interest of the user.

32. The apparatus of claim 31, wherein the address is web address.

33. An apparatus for updating a registered word comprising:

a first memory unit and a second memory unit to store registered words;

a document extraction unit accessing a server through a network and extracting a document of a field of interest of a user in each predetermined extraction period;

a registered word extraction unit to extract a registered word stored in the first memory unit in the document extracted during the extraction period; and

a control unit to re-register in the second memory unit, a registered word stored in the first memory unit, based on at least one of the user'"'"'s usage frequency of the registered word, the appearance frequency and the changed appearance frequency of the registered word in the document.

34. The apparatus of claim 33, wherein the control unit comprises:

a usage frequency calculation unit calculating the user'"'"'s usage frequency of the registered word by counting the usage frequency of the registered word whenever the registered word stored in the first memory unit is used by the user;

an appearance frequency calculation unit calculating the appearance frequency of the registered word stored in the first memory, in the document through the registered word extraction unit;

a changed appearance frequency value calculation unit calculating the changed appearance frequency value of the registered word stored in the first memory unit from the appearance frequency of the registered word; and

a memory control unit re-registering in the second memory unit, a registered word stored in the first memory unit, based on any one of the user'"'"'s usage frequency, the appearance frequency, and the changed appearance frequency value of the registered word stored in the first memory unit.

35. An apparatus for updating a dictionary of a voice recognition system providing a service corresponding to a voice command of a user, with an unregistered word not registered in the dictionary, the apparatus comprising:

a document extraction unit accessing a server through a network and extracting a document of interest of a user in each predetermined extraction period;

an unregistered word extraction unit extracting candidate unregistered words existing in the extracted document according to predetermined unregistered word extraction rules, and based on appearance frequency information of the candidate unregistered words in the document, extracting unregistered words among the candidate unregistered words;

an unregistered word verification unit verifying the unregistered words with usage examples of the unregistered words extracted through the server, and allocating semantic information of the verified unregistered words;

a memory unit storing the unregistered words and the semantic information of the unregistered words;

a voice recognition control unit controlling a voice recognition model and a natural language processing model in order to reflect an unregistered word stored in the memory unit; and

a registration unit registering the unregistered words and the semantic information of the unregistered words in a predetermined location of the memory unit.