System and method of controlling external apparatus connected with device

• US 10,720,046 B2
• Filed: 08/08/2019
• Issued: 07/21/2020
• Est. Priority Date: 09/10/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Shin, Hang-sik, Ko, Jae-woo, Park, Se-jun
• Primary Examiner(s): Small, Naomi J
• Application Number:
  US16/535,748
• Publication Number:
  US 20190362624A1
• Time in Patent Office: 348 Days
• Field of Search: 340 125
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: H04L 67/303 Terminal profiles

Abstract

A method of controlling an external apparatus includes receiving a user input information; obtaining apparatus information regarding a plurality of external apparatuses; selecting one or more external apparatuses, from the plurality of external apparatuses, which is communicable with and controllable based on the user input information; generating a control information for controlling the one or more external apparatuses based on a user'"'"'s input and the apparatus information; and transmitting a control command to the one or more external apparatuses, the control command being generated based on the received control information.

Claims

1. A method of controlling external devices, the method comprising:

receiving, from a server, identification information of a plurality of external devices and capability information regarding capabilities that the plurality of external devices are able to perform;

receiving, from an electronic device distinct from the plurality of external devices, a single user voice command input by a user to the electronic device;

obtaining a user intent regarding the received single user voice command based on the received single user voice command;

generating control information for controlling at least two external devices, from among the plurality of external devices, based on the obtained user intent regarding the single user voice command; and

outputting the generated control information for controlling the at least two external devices.

2. The method of claim 1, wherein:

the generated control information comprises capability information for controlling operations of the at least two external devices.

3. The method of claim 1, wherein:

the single user voice command is a phrase which is input into the electronic device by a user voice, and comprises a combinational instruction, and

the combinational instruction is directed to the electronic device and comprises an action, a time, and a condition.

4. The method of claim 1, wherein the receiving the identification information and the capability information comprises receiving the identification information and the capability information based on user input information, the user input information being received from the electronic device and input by the user to the electronic device.

5. The method of claim 1, wherein the received identification information and the received capability information are received separately from and independently of the receiving the single user voice command.

6. The method of claim 5, wherein:

the received single user voice command does not include identifications of each of the at least two external devices.

7. The method of claim 1, wherein the electronic device and the plurality of external devices are connected to a same network via a home gateway.

8. The method of claim 1, wherein the receiving the single user voice command comprises receiving, by the server from the electronic device via an Internet connection, the single user voice command input by the user to the electronic device.

9. The method of claim 1, further comprising, based on an external device being unable to be identified based on the received single user voice command:

requesting additional user input information from the electronic device; and

receiving, from the electronic device, an additional user voice information input by the user to the electronic device.

10. The method of claim 1, wherein the generating the control information for controlling the at least two external devices comprises generating the control information for control commands controlling the at least two external devices in a predetermined order.

11. A non-transitory computer-readable recording medium having recorded thereon a computer program which, when executed by a processor, causes the processor to execute the method of claim 1.

12. The method of claim 1, wherein the obtaining the user intent regarding the received single user voice command based on the received single user voice command comprises performing natural language processing on the received single user voice command.

13. The method of claim 1, wherein the at least two external devices of the control information are from among the plurality of external devices with respect to which the identification information is received.

14. A first server comprising:

a memory for storing instructions; and

a processor configured to execute the instructions to;

obtain, from a second server, identification information of a plurality of external devices and capability information regarding capabilities that the plurality of external devices are able to perform,

obtain, from an electronic device distinct from the plurality of external devices, a single user voice command input by a user to the electronic device,

obtain a user intent regarding the obtained single user voice command based on the obtained single user voice command,

generate control information for controlling at least two external devices, based on the obtained user intent regarding the single user voice command, and

output the generated control information for controlling the at least two external devices.

15. The first server of claim 14, wherein:

the single user voice command is a phrase which is input into the electronic device by a user voice, and comprises a combinational instruction, and

the combinational instruction is directed to the electronic device and comprises an action, a time, and a condition.

16. The first server of claim 14, wherein the processor is configured to obtain the identification information and the capability information based on user input information, the user input information being received from the electronic device and input by the user of the electronic device.

17. The first server of claim 14, wherein the obtained identification information and the obtained capability information are obtained separately from and independently of the obtaining the single user voice command.

18. The first server of claim 17, wherein:

the obtained single user voice command does not include identifications of each of the at least two external devices.

19. The first server of claim 14, wherein the electronic device and the plurality of external devices are connected to a same network via a home gateway.

20. The first server of claim 14, wherein, based on an external device being unable to be identified based on the obtained single user voice command, the processor is further configured to:

request additional user input information from the electronic device; and

obtain, from the electronic device, an additional user voice information input by the user to the electronic device.

21. The first server of claim 14, wherein the processor is configured to generate the control information for control commands controlling the at least two external devices in a predetermined order.

22. The first server of claim 14, wherein the processor is configured to obtain the user intent regarding the obtained single user voice command based on the obtained single user voice command by performing natural language processing on the obtained single user voice command.

23. A system for controlling external devices used by a user, the system comprising:

a first server comprising;

a memory for storing instructions; and

a processor configured to execute the instructions to;

obtain, from a second server, first identification information of a plurality of external devices and first capability information regarding capabilities that the plurality of external devices are able to perform,

obtain, from an electronic device distinct from the plurality of external devices, a single user voice command input by a user to the electronic device,

obtain a user intent regarding the obtained single user voice command based on the obtained single user voice command,

generate control information for controlling at least two external devices, from among the plurality of external devices, based on the obtained user intent regarding the single user voice command, and

output the generated control information for controlling the at least two external devices, and

the electronic device comprising;

a memory for storing instructions; and

a processor configured to execute the instructions to;

receive the single user voice command input,

transmit the single user voice command to the first server.

24. The system of claim 23, wherein the processor of the first server is configured to execute the instructions to:

identify the at least two external devices, from among the plurality of external devices, based on the obtained user intent regarding the single user voice command and the first identification information of the plurality of external devices; and

generate the control information for control commands controlling the identified at least two external devices, based on the obtained user intent regarding the single user voice command, second capability information of the identified at least two external devices, and second identification information of the identified at least two external devices.

25. A non-transitory computer-readable recording medium having recorded thereon a computer program which, when executed by a processor, causes the processor to:

obtain, from a server, first identification information of a plurality of external devices and first capability information regarding capabilities that the plurality of external devices are able to perform;

obtain, from an electronic device distinct from the plurality of external devices, a single user voice command input by a user to the electronic device;

obtain a user intent regarding the received single user voice command based on the received single user voice command;

generate control information for controlling at least two external devices, based on the obtained user intent regarding the single user voice command; and

output the generated control information for controlling the at least two external devices.

26. The non-transitory computer-readable recording medium of claim 25, wherein the computer program causes the processor to:

identify the at least two external devices, from among the plurality of external devices, based on the obtained user intent regarding the single user voice command and the first identification information of the plurality of external devices; and

generate the control information for control commands controlling the identified at least two external devices, based on the obtained user intent regarding the single user voice command, second capability information of the identified at least two external devices, and second identification information of the identified at least two external devices.