Device and method for performing functions

• US 10,768,892 B2
• Filed: 09/11/2018
• Issued: 09/08/2020
• Est. Priority Date: 07/31/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Bang, Han-min, Choi, Hyok-sung
• Primary Examiner(s): Belousov, Andrey
• Application Number:
  US16/127,943
• Publication Number:
  US 20190012142A1
• Time in Patent Office: 728 Days
• Field of Search: 715728
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06F 9/5044 considering hardware capabilities
• CPC Class Codes: G06F 9/5055 considering software capabilities, i.e. software resources associated or available to the machine
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 1/7243 with interactive means for internal management of messages
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00

Abstract

Provided is a device including a display, an audio inputter, and a controller. The display displays at least one screen page of an application that is being executed. The audio inputter receives a voice command of a user. The controller performs an operation corresponding to the voice command by using screen page transition information for transition between application screen pages corresponding to the voice command, which is obtained from information about user interface (UI) elements included in the application screen pages of the application. Each of the UI elements performs a predetermined function when selected by the user.

Claims

1. A device comprising:

a communicator configured to receive service information from at least two Internet of Things (IoT) devices;

a memory configured to store the service information received from the at least two IoT devices; and

an inputter configured to receive a user input comprising a command phrase that requests performance of at least two predetermined functions; and

a controller configured to;

analyze the user input and the service information of the at least two IoT devices stored in the memory;

determine the at least two predetermined functions based on the analyzing of the user input and the service information of the at least two IoT devices,

select at least two IoT devices for performing each of the at least two predetermined functions based on the determined at least two predetermined functions,

transmit a first execution command to a first IoT device among the selected at least two IoT devices, and

transmit a second execution command to a second IoT device among the selected at least two IoT devices,

wherein the first execution command comprises information related to execution of a first predetermined function performed by the first IoT device and information related to transmitting result data of the execution of the first predetermined function to the second IoT device as an input parameter of a second predetermined function performed by the second IoT device,

wherein the second execution command comprises information related to execution of the second predetermined function performed by the second IoT device and information related to receiving the result data of the execution the first predetermined function as an input parameter of the second predetermined function, and

wherein the service information received from the at least two IoT devices comprises a list of predetermined functions provided by the at least two IoT devices.

2. The device of claim 1, wherein the service information comprises at least one of predetermined functions provided by the at least two IoT devices, device identification information, sensor data collected by a sensor and operation information of the at least two IoT devices.

3. The device of claim 1, wherein the communicator is further configured to provide the user input to a server and receive text data generated by analyzing the user input from the server.

4. The device of claim 1, wherein the user input is at least one of a voice input or a text input.

5. The device of claim 1, wherein the communicator is further configured to receive a periodically updated list of predetermined functions from the at least two IoT devices.

6. The device of claim 1, wherein the communicator is further configured to connect to the at least two IoT devices through a communication link.

7. A method of performing a function by a device, the method comprising:

receiving service information from at least two Internet of Things (IoT) devices;

storing the service information received from the at least two IoT devices into a memory;

receiving a user input comprising a command phrase that requests performance of at least two predetermined functions;

analyzing the user input and the service information of the at least two IoT devices stored in the memory;

determining the at least two predetermined functions based on the analyzing of the user input and the service information of the at least two IoT devices;

selecting at least two IoT devices for performing each of the at least two predetermined functions based on the determined at least two predetermined functions;

transmitting a first execution command to a first IoT device among the selected at least two IoT devices; and

transmitting a second execution command to a second IoT device among the selected at least two IoT devices,

wherein the first execution command comprises information related to execution of a first predetermined function performed by the first IoT device and information related to transmitting result data of the execution of the first predetermined function to the second IoT device as an input parameter of a second predetermined function performed by the second IoT device,

wherein the second execution command comprises information related to execution of the second predetermined function performed by the second IoT device and information related to receiving the result data of the execution the first predetermined function as an input parameter of the second predetermined function,

wherein the service information received from the at least two IoT devices comprises a list of predetermined functions provided by the at least two IoT devices.

8. The method of claim 7, wherein the service information comprises at least one of predetermined functions provided by the at least two IoT devices, device identification information, sensor data collected by a sensor and operation information of the at least two IoT devices.

9. The method of claim 7, further comprising providing the user input to a server and receiving text data generated by analyzing the user input from the server.

10. The method of claim 7, wherein the user input is at least one of a voice input or a text input.

11. The method of claim 7, wherein the receiving of the service information from the at least two IoT devices comprises receiving a periodically updated list of predetermined function from the at least two IoT devices.

12. The method of claim 7, further comprising connecting to the at least two IoT devices through a communication link.

13. A non-transitory computer-readable recording medium having recorded thereon a program for performing the method of claim 7.