Interactive system, display apparatus, and controlling method thereof

• US 9,886,952 B2
• Filed: 01/05/2015
• Issued: 02/06/2018
• Est. Priority Date: 01/22/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Choi, Chan-hee, Park, Kyung-mi, Hwang, Kwang-il
• Primary Examiner(s): Hudspeth, David
• Assistant Examiner(s): Patel, Shreyans
• Application Number:
  US14/589,313
• Publication Number:
  US 20150206530A1
• Time in Patent Office: 1,128 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

An interactive system, a display apparatus, and a controlling method are provided. The display apparatus includes an input which receives an uttered voice of a user; a communicator which transmits a voice signal of the uttered voice to a voice recognition apparatus; a voice recognizer which performs a voice recognition process with the uttered voice; and a controller which determines first or second voice information which has a reliability value greater than or equal to a preset threshold value among a reliability value of the first voice information, and a reliability value of the second voice information to be an execution command of the uttered voice. Therefore, if the display apparatus and an external apparatus simultaneously recognize the uttered voice of the user, the display apparatus selects a voice recognition result proximate to an intent of the user from two voice recognition results.

Claims

1. A display apparatus comprising:

an input configured to receive an uttered voice of a user;

a communicator configured to transmit a voice signal of the uttered voice to a voice recognition apparatus, which is external to the display apparatus, and receive a recognition result of the voice signal from the voice recognition apparatus;

a voice recognizer configured to perform a voice recognition process with respect to the uttered voice; and

a controller configured to;

compare a reliability of a first voice information recognized by the voice recognizer to a preset threshold value and compare a reliability value of a second voice information of the recognition result received from the voice recognition apparatus to the preset threshold value, wherein the first voice information and the second voice information are text information into which the voice signal of the uttered voice is converted; and

in response to the reliability value of the first voice information and the reliability value of the second voice information being greater than or equal to the preset threshold value, compare a length of a character string of the text information of the first voice information and a length of a character string of the text information of the second voice information and determine one from among the first voice information and the second voice information, which has a longer character string, to be an execution command.

2. The display apparatus of claim 1, wherein the controller is configured to calculate a similarity value of pronunciation strings from the text information of the first voice information and the text information of the second voice information based on a predefined similarity algorithm when the length of the character string of the text information of the first voice information and the length of the character string of the text information of the second voice information are equal, determine the first voice information as an execution command when the calculated similarity value is greater than or equal to a preset threshold value, and determine the second voice information to be an execution command when the calculated similarity value is lower than the preset threshold value.

3. The display apparatus of claim 2, wherein the controller is further configured to, when the execution command of the uttered voice is determined, determine a domain based on one from among the first voice information and the second voice information determined to be the execution command and is configured to determine the one from among the first voice information and the second voice information determined to be the execution command to be a final execution command based on a condition that the determined domain and a domain of a current operation mode of the display apparatus correspond to each other.

4. The display apparatus of claim 1, wherein the controller is further configured to, when the reliability values of the first voice information and the reliability values of the second voice information are lower than the preset threshold value, determine a domain of the first voice information and a domain of the second voice information, apply a weight to one from among the reliability value of the first voice information and the reliability value of the second voice information, which has a domain corresponding to the domain of a current operation mode of the display apparatus, and determine the one from among the first voice information and the second voice information to which the weight is applied to be an execution command based on a condition that the reliability value of the one from among the first voice information and the second voice information to which the weight is applied is greater than or equal to the preset threshold value.

5. The display apparatus of claim 1, further comprising:

a storage configured to store control commands corresponding to a plurality of pieces of voice information; and

a display configured to display a voice recognition result of response information received from an interactive server,

wherein the controller is further configured to, when the first voice information is recognized by the voice recognizer and is determined to be the execution command, perform a control operation according to a control command corresponding to the first voice information among the control commands stored in the storage and, in response to the second voice information being received from the voice recognition apparatus and being determined to be the execution command, transmit the second voice information to the interactive server through the communicator to receive response information related to the second voice information from the interactive server.

6. A method of controlling a display apparatus, the method comprising:

receiving an uttered voice of a user;

performing a voice recognition process with respect to the uttered voice to calculate a reliability value of first voice information;

transmitting a voice signal of the uttered voice of the user to a voice recognition apparatus, which is external to the display apparatus, and receiving a voice recognition result of the voice signal from the voice recognition apparatus;

in response to the voice recognition result of the uttered voice being received from the voice recognition apparatus, comparing the reliability value of the first voice information to a preset threshold value and comparing a reliability value of second voice information received in the voice recognition result to the preset threshold value, wherein the first voice information and the second voice information are text information into which a voice signal of the uttered voice is converted; and

in response to the reliability value of the first voice information and the reliability value of the second voice information being greater than or equal to the preset threshold value, comparing a length of a character string of the text information of the first voice information and a length of a character string of the second voice information and determining one from among the first voice information and the second voice information, which has a longer character string, to be an execution command.

7. The method of claim 6, wherein when the length of the character string of the first voice information and the length of the character string of the second voice information are the same, a similarity value of pronunciation strings of the text information of the first voice information and the text information of the second voice information is calculated based on a pre-stored similarity algorithm, when the calculated reliability value is greater than or equal to a preset threshold value, the first voice information is determined to be an execution command, and when the calculated similarity value is lower than the preset threshold value, the second voice information is determined to be an execution command.

8. The method of claim 7, further comprising:

when the execution command of the uttered voice is determined, determining a domain based on the first voice information or the second voice information determined as the execution command; and

determining the first or second voice information determined to be the execution command as a final execution command according to a condition that the determined domain corresponds to a domain of a current operation mode of the display apparatus.

9. The method of claim 6, further comprising:

when the reliability value of the first voice information and the reliability value of the second voice information are lower than the preset threshold value, determining domains of the first voice information and the second voice information; and

applying a weight to the reliability value of the first voice information or the second voice information which has a domain corresponding to a domain of a current operation mode of the display apparatus,

wherein the first voice information or second voice information to which the weight is applied is determined to be an execution command according to a condition that the reliability value of the first voice information or the second voice information to which the weight is applied is greater than or equal to the preset threshold value.

10. The method of claim 6, further comprising:

when the first voice information is determined to be the execution command, performing a control operation according to a control command related to the first voice information among a plurality of pre-stored control commands, and when the second voice information is determined to be the execution command, receiving response information related to the second voice information from an interactive server and displaying a voice recognition result related to the received response information.

11. An interactive system comprising:

a display apparatus configured to, when an uttered voice of a user is input, perform a voice recognition process based on a voice signal of the uttered voice and transmit the voice signal to a voice recognition apparatus, which is external to the display apparatus; and

the voice recognition apparatus configured to perform a voice recognition process based on the voice signal received from the display apparatus and transmit a voice recognition result of the voice recognition to the display apparatus,

wherein when the voice recognition result is received from the voice recognition apparatus, the display apparatus compares a reliability value of a first voice information to a preset threshold value and a reliability value of a second voice information to the preset threshold value, wherein the first voice information and the second voice information are text information into which a voice signal of the uttered voice is converted, and in response to the reliability value of the first voice information and the reliability value of the second voice information is greater than or equal to the threshold value, the display apparatus compares a length of a character string of the text information of the first voice information and a length of a character string of the text information of the second voice information and determines the first voice information or the second voice information which has a longer character string to be an execution command.

12. The interactive system of claim 11, wherein the display apparatus calculates a similarity value of pronunciation strings of the text information of the first voice information and the text information of the second voice information based on a predefined similarity algorithm when the lengths of the character strings are the same, determines the first voice information to be an execution command when the calculated similarity value is greater than or equal to a preset threshold value, and determines the second voice information to be an execution command in response to the calculated similarity value being lower than the threshold value.

13. The interactive system of claim 12, wherein the display apparatus determines a domain based on the first voice information or the second voice information determined to be the execution command when the execution command of the uttered voice is determined and determines the first voice information or the second voice information determined to be the execution command as a final execution command according to whether the determined domain corresponds to a domain of a current operation mode of the display apparatus.

14. The interactive system of claim 11, wherein when the reliability value of the first voice information and the reliability value of the second voice information is lower than the threshold value, the display apparatus determines a domain of the first voice information and a domain of the second voice information, apply a weight to the reliability value of the first voice information or the second voice information which has a domain corresponding to a domain of a current operation mode of the display apparatus, and determines the first voice information or the second voice information to which the weight is applied to be an execution command upon a condition that the reliability value of the first voice information or the second voice information to which the weight is applied is greater than or equal to the threshold value.

15. The interactive system of claim 11, wherein the display apparatus performs a control operation according to a control command related to the first voice information when the first voice information is determined to be the execution command, and transmits the second voice information to an interactive server to receive response information related to the second voice information and displays a voice recognition result related to the received response information when the second voice information is determined to be the execution command.

16. A method of controlling a display apparatus, the method comprising:

receiving an uttered voice of a user at the display apparatus;

converting the uttered voice to a voice signal, and generating first voice information;

calculating a reliability value of the first voice information;

transmitting the voice signal to a voice recognition apparatus, which is external to the display apparatus;

generating, by the voice recognition apparatus, second voice information;

receiving the second voice information from the voice recognition apparatus;

calculating a reliability value of the second voice information;

comparing the reliability value of the first voice information and the reliability value of the second voice information with a threshold value, wherein the first voice information and the second voice information are text information into which the voice signal of the uttered voice are converted;

in response to the reliability value of the first voice information and the reliability value of the second voice information being equal to or greater than the threshold value, comparing a length of a character string of the text information of the first voice information and a length of a character string of the text information of the second voice information; and

determining voice information having a character string of a longer length from among the length of the character string of the text information of the first voice information and the length of the character string of the text information of the second voice information to be an execution command.