System and method of controlling external apparatus connected with device

• US 10,460,597 B2
• Filed: 11/02/2017
• Issued: 10/29/2019
• Est. Priority Date: 09/10/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Shin, Hang-sik, Ko, Jae-woo, Park, Se-jun
• Primary Examiner(s): Small, Naomi J
• Application Number:
  US15/801,417
• Publication Number:
  US 20180061223A1
• Time in Patent Office: 726 Days
• Field of Search: 340 125, 340 582
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: H04L 67/303 Terminal profiles

Abstract

A method of controlling an external apparatus includes receiving a user input information; obtaining apparatus information regarding a plurality of external apparatuses; selecting one or more external apparatuses, from the plurality of external apparatuses, which is communicable with and controllable based on the user input information; generating a control information for controlling the one or more external apparatuses based on a user'"'"'s input and the apparatus information; and transmitting a control command to the one or more external apparatuses, the control command being generated based on the received control information.

Claims

1. A method of controlling external devices, the method comprising:

receiving, from a server, first identification information of a plurality of external devices and first capability information regarding capabilities that the plurality of external devices are able to perform;

receiving, from an electronic device distinct from the plurality of external devices, a single user voice command input by a user to the electronic device;

obtaining a user intent regarding the received single user voice command based on the received single user voice command;

identifying at least two external devices, from among the plurality of external devices, based on the obtained user intent regarding the single user voice command and the first identification information of the plurality of external devices;

generating control information for control commands controlling the identified at least two external devices, based on the obtained user intent regarding the single user voice command, second capability information of the identified at least two external devices, and second identification information of the identified at least two external devices; and

outputting the generated control information for the control commands controlling the identified at least two external devices.

2. The method of claim 1, wherein:

the generated control information comprises capability information for controlling operations of the identified at least two external devices, and

the capability information is used by the identified at least two external devices in a predetermined order.

3. The method of claim 1, wherein:

the single user voice command is a phrase which is input into the electronic device by a user voice, and comprises a combinational instruction, and

the combinational instruction is directed to the electronic device and comprises an action, a time, and a condition.

4. The method of claim 1, wherein the receiving the first identification information and the capability information comprises receiving the first identification information and the capability information based on user input information, the user input information being received from the electronic device and input by the user to the electronic device.

5. The method of claim 1, wherein the received first identification information and the received capability information are received separately from and independently of the receiving the single user voice command.

6. The method of claim 5, wherein:

the received single user voice command does not include identifications of each of the identified at least two external devices; and

the identifying the at least two external devices comprises analyzing the received single user voice command to determine the at least two external devices corresponding to the received single user voice command.

7. The method of claim 1, wherein the electronic device and the plurality of external devices are connected to a same network via a home gateway.

8. The method of claim 1, wherein the receiving the single user voice command comprises receiving, by the server from the electronic device via an Internet connection, the single user voice command input by the user to the electronic device.

9. The method of claim 1, wherein the identifying the at least two external devices comprises, based on the server being unable to identify an external device based on the received single user voice command:

requesting additional user input information from the electronic device;

receiving, from the electronic device, an additional user voice information input by the user to the electronic device; and

identifying the at least two external devices based on the received additional user voice information.

10. The method of claim 1, wherein the generating the control information for the control commands controlling the identified at least two external devices comprises generating the control information for the control commands controlling the identified at least two external devices in a predetermined order.

11. A non-transitory computer-readable recording medium having recorded thereon a computer program which, when executed by a processor, causes the processor to execute the method of claim 1.

12. The method of claim 1, wherein the obtaining the user intent regarding the received single user voice command based on the received single user voice command comprises performing natural language processing on the received single user voice command.

13. A first server comprising:

a memory for storing instructions; and

a processor configured to execute the instructions to;

obtain, from a second server, first identification information of a plurality of external devices and first capability information regarding capabilities that the plurality of external devices are able to perform,

obtain, from an electronic device distinct from the plurality of external devices, a single user voice command input by a user to the electronic device,

obtain a user intent regarding the obtained single user voice command based on the obtained single user voice command,

identify at least two external devices, from among the plurality of external devices, based on the obtained user intent regarding the single user voice command and the first identification information of the plurality of external devices,

generate control information for control commands controlling the identified at least two external devices, based on the obtained user intent regarding the single user voice command, second capability information of the identified at least two external devices, and second identification information of the identified at least two external devices, and

output the generated control information for the control commands controlling the identified at least two external devices.

14. The first server of claim 13, wherein:

the single user voice command is a phrase which is input into the electronic device by a user voice, and comprises a combinational instruction, and

the combinational instruction is directed to the electronic device and comprises an action, a time, and a condition.

15. The first server of claim 13, wherein the processor is configured to obtain the first identification information and the first capability information based on user input information, the user input information being received from the electronic device and input by the user of the electronic device.

16. The first server of claim 13, wherein the obtained first identification information and the obtained first capability information are obtained separately from and independently of the obtaining the single user voice command.

17. The first server of claim 16, wherein:

the obtained single user voice command does not include identifications of each of the identified at least two external devices; and

the processor is configured to identify the at least two external devices by analyzing the obtained single user voice command to determine the at least two external devices corresponding to the obtained single user voice command.

18. The first server of claim 13, wherein the electronic device and the plurality of external devices are connected to a same network via a home gateway.

19. The first server of claim 13, wherein, based on the first server being unable to identify an external device based on the obtained single user voice command, the processor is configured to identify the at least two external devices by:

requesting additional user input information from the electronic device;

obtaining, from the electronic device, an additional user voice information input by the user to the electronic device; and

identifying the at least two external devices based on the obtained additional user voice information input.

20. The first server of claim 13, wherein the processor is configured to generate the control information for control commands controlling the identified at least two external devices in a predetermined order.

21. The first server of claim 13, wherein the processor is configured to obtain the user intent regarding the obtained single user voice command based on the obtained single user voice command comprises performing natural language processing on the obtained single user voice command.