Display apparatus, interactive system, and response information providing method

• US 9,412,368 B2
• Filed: 06/14/2013
• Issued: 08/09/2016
• Est. Priority Date: 07/03/2012
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Heo, Hye-hyun, Kim, Ki-suk, Son, Hae-rim
• Primary Examiner(s): VO, HUYEN X
• Application Number:
  US13/918,034
• Publication Number:
  US 20140012585A1
• Time in Patent Office: 1,152 Days
• Field of Search: 704/235, 704/251, 704/255, 704/257, 704/270, 704/270.1, 704 1- 10, 704/250
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: H04H 60/72 using electronic programme guides [EPG] focusing on identifying broadcast space time H04H60/39 ; menu type display of EPG in television receivers H04N21/47
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/2393 involving handling client requests scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/4828 for searching program descriptors retrieval of video data G06F16/739

Abstract

A display apparatus includes a voice collecting device which collects a user voice, a communication device which performs communication with an interactive server, and a control device which, when response information corresponding to the user voice sent to the interactive server is received from the interactive server, controls to perform a feature corresponding to the response information, and the control device controls the communication device to receive replacement response information, related to the user voice, through a web search and a social network service (SNS).

Claims

1. A display apparatus comprising:

a voice collecting device configured to collect a user voice;

a communication device configured to communicate with an interactive server, a web server, and a social network service (SNS) server; and

a control device configured to;

in response to response information corresponding to the user voice that is sent to the interactive server, being received from the interactive server, perform a feature corresponding to the response information; and

in response to a non-provisionable message for the response information corresponding to the user voice, being received from the interactive server, as it is not possible for the interactive server to provide the response information corresponding to the user voice, control the communication device to receive replacement response information, related to the user voice, through a web search and an SNS via the web server and the SNS server.

2. The display apparatus of claim 1, further comprising an output device,

wherein the control device is further configured to, in response to an utterance element that is extracted from the user voice and sent to the web server and the SNS server, and the non-provisionable message, being received from the interactive server:

control the communication device to receive, from the web server, a result of the web search using the utterance element, and receive, from the SNS server, a user message responding to the utterance element; and

control the output device to output the result of the web search and the user message.

3. The display apparatus of claim 2, wherein the control device is further configured to, in response to a user command for performing the web search being received:

control the communication device to receive the result of the web search from the web server; and

control the output device to output the result of the web search.

4. The display apparatus of claim 2, wherein the control device is further configured to, in response to a user command for an SNS being received:

generate a response request message requesting a response to the utterance element;

control the communication device to transmit the response request message to the SNS server providing the SNS;

control the communication device to receive, from the SNS server, the user message responding to the response request message; and

control the output device to output the user message.

5. The display apparatus of claim 4, wherein, depending on the user command, the control device is further configured to:

control the communication device to transmit the response request message to an SNS account of a user;

control the communication device to receive the user message from another SNS account that is linked with the SNS account of the user; and

control the output device to output the user message.

6. The display apparatus of claim 4, wherein, depending on the user command, the control device is further configured to:

control the communication device to transmit the response request message to an SNS account that is selected among SNS accounts linked with another SNS account of a user;

control the communication device to receive the user message from the selected SNS account; and

control the output device to output the user message.

7. The display apparatus of claim 4, wherein, depending on the user command, the control device is further configured to:

control the communication device to transmit the response request message to a predetermined public SNS account;

control the communication device to receive the user message from an SNS account that is linked with the predetermined public SNS account; and

control the output device to output the user message.

8. The display apparatus of claim 4, wherein the control device is further configured to, in response to the communication device receiving the user message control the output device to output notice information indicating that the user message is received.

9. The display apparatus of claim 1, wherein the interactive server comprises:

a first server configured to convert a digital signal corresponding to the user voice into text information; and

a second server configured to generate the response information based on the text information,

wherein the control device is further configured to;

convert the user voice into the digital signal;

control the communication device to transmit the digital signal to the first server; and

in response to the text information being received from the first server, control the communication device to transmit the text information to the second server, and receive the response information from the second server.

10. The display apparatus of claim 9, wherein the control device is further configured to, in response to the user voice comprising utterance elements related to an electronic program guide or a control operation of the display apparatus:

output the response information; and

/or

execute a function corresponding to the user voice based on the response information.

11. A response information providing method of providing response information appropriate to a user voice in a display apparatus, the response information providing method comprising:

collecting the user voice;

transmitting the collected user voice to an interactive server;

determining whether response information corresponding to the user voice is received from the interactive server;

performing a feature corresponding to the user voice based on the response information in response to the determining that the response information corresponding to the user voice is received; and

in response to a non-provisionable message for the response information corresponding to the user voice, being received from the interactive server, as it is not possible for the interactive server to provide the response information corresponding to the user voice, receiving and outputting replacement response information related to the user voice through a web search and a social network service (SNS) via a web server and an SNS server.

12. The response information providing method of claim 11, wherein the performing further comprises, in response to an utterance element that is extracted from the user voice and sent to the web server and the SNS server, and the non-provisionable message, being received from the interactive server:

receiving, from the web server, and outputting a result of the web search using the utterance element; and

receiving, from the SNS server, and outputting a user message responding to the utterance element.

13. The response information providing method of claim 12, wherein the performing further comprises, in response to a user command for performing the web search being received, receiving, from the web server, and outputting the result of the web search.

14. The response information providing method of claim 12, wherein the performing further comprises, in response to a user command for an SNS being received:

generating a response request message requesting a response to the utterance element;

transmitting the response request message to the SNS server providing the SNS; and

receiving, from the SNS server, and outputting the user message responding to the response request message.

15. The response information providing method of claim 14, wherein the performing further comprises, depending on the user command:

transmitting the response request message to an SNS account of a user; and

receiving, from another SNS account that is linked with the SNS account of the user, and outputting the user message.

16. The response information providing method of claim 14, wherein the performing further comprises, depending on the user command:

transmitting the response request message to an SNS account that is selected among SNS accounts linked with another SNS account of a user; and

receiving, from the selected SNS account, and outputting the user message.

17. The response information providing method of claim 14, wherein the performing further comprises, depending on the user command:

transmitting the response request message to a predetermined public SNS account; and

receiving from a SNS account that is linked with the predetermined public SNS account, and outputting the user message.

18. The response information providing method of claim 14, wherein the performing further comprises outputting notice information indicating that the user message is received in response to the receiving the user message.

19. The response information providing method of claim 11, wherein the interactive server comprises:

a first server converting a digital signal corresponding to the user voice into text information; and

a second server generating the response information based on the text information,

wherein the transmitting comprises;

converting the user voice into the digital signal;

transmitting the digital signal to the first server;

receiving the text information from the first server;

transmitting the text information to the second server; and

receiving the response information from the second server.

20. The response information providing method of claim 19, wherein the performing further comprises, in response to the user voice comprising utterance elements related to an electronic program guide or a control operation of the display apparatus:

outputting the response information; and

/or

executing a function corresponding to the user voice based on the response information.

21. A display apparatus comprising:

a voice collecting device configured to;

collect a user voice; and

remove noise from the collected user voice to generate a filtered user voice;

a photographing device configured to photograph a face of a user to generate a face image;

a communication device configured to;

transmit the filtered user voice and the face image to an interactive server; and

receive, from the interactive server, response information appropriate to the filtered user voice and the face image; and

a control device configured to;

perform an operation corresponding to the filtered user voice and the face image based on the response information; and

in response to a non-provisionable message for the response information corresponding to the filtered user voice, being received from the interactive server, as it is not possible for the interactive server to provide the response information corresponding to the filtered user voice, control the communication device to receive replacement response information related to the filtered user voice through a web search and an SNS via a web server and an SNS server.

22. The display apparatus of claim 21, wherein the control device is further configured to, in response to an utterance element that is extracted from the user voice and sent to the web server and the SNS server, and the non-provisionable message being received from the interactive server, control the communication device to receive, from the web server, a result of web search using the utterance element, and receive, from the SNS server, a user message responding to the utterance element.

23. An interactive system providing response information appropriate to a user voice, the interactive system comprising:

a display apparatus configured to recognize the user voice; and

an interactive server configured to, in response to the user voice being received from the display apparatus;

convert the user voice into text information;

analyze the text information; and

transmit, to the display apparatus, response information corresponding to the user voice based on the analyzed text information,

wherein the display apparatus is further configured to;

perform a function corresponding to the response information in response to the response information being received from the interactive server;

in response to a non-provisionable message for the response information corresponding to the user voice, being received from the interactive server, as it is not possible for the interactive server to provide the response information corresponding to the user voice, receive replacement response information related to the user voice through a web search and an SNS via the web server and the SNS server; and

perform a function related to the user voice based on the replacement response information.