Method and apparatus for recognizing speech according to dynamic display

• US 9,269,356 B2
• Filed: 07/28/2010
• Issued: 02/23/2016
• Est. Priority Date: 07/31/2009
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Han, Ick-Sang, Cho, Jeong-Mi
• Primary Examiner(s): Shah, Paras D
• Application Number:
  US12/844,898
• Publication Number:
  US 20110029301A1
• Time in Patent Office: 2,036 Days
• Field of Search: 704/9, 704/10, 704/231, 704/235, 704/236, 704/251, 704/252, 704/255, 704/257, 701/200, 701/211, 701/212, 701/208, 701/213, 701/1
• US Class Current: 1/1
• CPC Class Codes: B60R 16/0373 Voice control in general G10L
• CPC Class Codes: G01C 21/3608 using speech input, e.g. using speech recognition
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G09G 2340/04 Changes in size, position or resolution of an image
• CPC Class Codes: G09G 2340/045 Zooming at least part of an image, i.e. enlarging it or shrinking it
• CPC Class Codes: G09G 3/34 by control of light from an independent source
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A speech recognition apparatus and method that can improve speech recognition rate and recognition speed by reflecting information for dynamic display, are provided. The speech recognition apparatus generates a display variation signal indicating that variations have occurred on a screen and creates display information about the varied screen. The speech recognition apparatus adjusts a word weight for at least one word related to the varied screen and a domain weight for at least one domain included in the varied screen, according to the display variation signal and the display information. The adjusted word weight and the adjusted domain weight are dynamically reflected in a language model that is used for speech recognition.

Claims

1. A speech recognition apparatus, comprising:

a display device configured to display data on a screen;

a controller configured to generate a display variation signal indicating that variations have occurred on a screen, and create display information about the varied screen; and

a speech recognizer configured to adjust a word weight for at least one word related to the varied screen and a domain weight for at least one domain included in the varied screen, according to the display variation signal and the display information, and perform speech recognition using a dynamic language model in which the adjusted word weight and the adjusted domain weight are reflected,

wherein the speech recognizer comprises a weight adjusting unit configured to

adjust word weights and domain weights according to variations of the display information, and

assign a greater domain weight to at least one domain having a greater portion overlapping a current screen and at least one domain having a shorter distance to a center of the current screen.

2. The speech recognition apparatus of claim 1, wherein the controller is further configured to generate the display variation signal in response to a user input signal.

3. The speech recognition apparatus of claim 2, wherein the user input signal is generated by at least one of cursor movement, magnification adjustment, and screen rotation.

4. The speech recognition apparatus of claim 1, wherein the controller is further configured to generate the display variation signal when a moving object appears on the varied screen.

5. The speech recognition apparatus of claim 1, wherein the speech recognizer is further configured to perform the speech recognition using a language model where greater word weights and greater domain weights are respectively assigned to words and domains associated with the current screen.

6. The speech recognition apparatus of claim 1, wherein the speech recognizer is further configured to:

recognize the at least one word related to the varied screen using a linguistic structure that provides at least one associated word according to coordinates and magnifications of screens;

recognize the at least one domain included in the varied screen using a domain structure configured to provide at least one associated domain according to coordinates and magnifications of screens; and

adjust the word weight for the at least one word and the domain weight for the at least one domain.

7. The speech recognition apparatus of claim 1, wherein the display information about the varied screen comprises at least one of a coordinate range of the varied screen, a magnification level, a word related to the varied screen, and a domain belonging to the varied screen.

8. The speech recognition apparatus of claim 1, wherein the speech recognizer further comprises:

a display information manager configured to manage the storage of display information that varies over time; and

a speech recognition engine configured to acquire the adjusted word weights and the adjusted domain weights when an utterance is input, and perform speech recognition using a language model generated dynamically according to the acquired word weights and the acquired domain weights.

9. The speech recognition apparatus of claim 8, wherein the display information manager is further configured to determine whether or not to store the display information based on at least one of a difference between a time at which the display variation signal has been received and a time at which the previous display variation signal has been received, variations in coordinate range between the current screen and the previous screen, and whether or not speech recognition has been performed on a screen.

10. The speech recognition apparatus of claim 8, wherein the weight adjusting unit is further configured to assign a greater word weight to at least one word included in the current screen and screens adjacent temporally and spatially to the current screen in comparison to words that are not included in the current screen and the adjacent screens.

11. The speech recognition apparatus of claim 8, wherein the weight adjusting unit is further configured to:

assign a greater domain weight to at least one domain included in the current screen and screens adjacent temporally and spatially to the current screen in comparison to domains not included in the current screen and the adjacent screens.

12. The speech recognition apparatus of claim 1, wherein the dynamic language model is calculated by interpolating a general language model that covers all domains, a domain language model defined for each domain, and a word language model in which word weight information is reflected.

13. A speech recognition method, comprising:

generating a display variation signal indicating that variations have occurred on a screen and creating display information about the varied screen;

adjusting a word weight for at least one word related to the varied screen and a domain weight for at least one domain included in the varied screen, according to the display variation signal and the display information; and

performing speech recognition using a dynamic language model in which the adjusted word weight and the adjusted domain weight are reflected,

wherein the adjusting of the word weight comprises

assigning a greater domain weight to at least one domain having a greater portion overlapping a current screen and at least one domain having a shorter distance to a center of the current screen.

14. The speech recognition method of claim 13, wherein the display variation signal is generated in response to a user input signal.

15. The speech recognition method of claim 14, wherein the user input signal is generated by at least one of cursor movement, magnification adjustment, and screen rotation.

16. The speech recognition method of claim 13, wherein the display variation signal is generated when a dynamically moving object appears on the varied screen.

17. The speech recognition method of claim 13, wherein the performing of the speech recognition comprises performing the speech recognition using a language model where greater word weights and greater domain weights are respectively assigned to words and domains associated with the current screen.

18. The speech recognition method of claim 13, wherein the adjusting of the word weight comprises assigning a greater word weight to at least one word included in the current screen and screens adjacent temporally and/or spatially to the current screen in comparison to words that are not included in the current screen and the adjacent screens.

19. The speech recognition method of claim 13, wherein the adjusting of the word weight further comprises:

assigning a greater domain weight to at least one domain included in the current screen and screens adjacent temporally and spatially to the current screen in comparison to domains not included in the current screen and the adjacent screens.

20. The speech recognition method of claim 13, wherein the dynamic language model is calculated by interpolating a general language model that covers all domains, a domain language model defined for each domain, and a word language model in which word weight information is reflected.

21. A speech recognition apparatus, comprising:

a display device configured to display words on a screen; and

a speech recognizer configured to perform speech recognition using a dynamic language model, and to dynamically adjust the dynamic language model such that words currently displayed on the screen have an increased probability of being recognized as speech when subsequently receiving an utterance from a user,

wherein the speech recognizer comprises a weight adjusting unit configured to

adjust word weights and domain weights according to variations of display information, and

assign a greater domain weight to at least one domain having a greater portion overlapping a current screen and at least one domain having a shorter distance to a center of the current screen.