Method and apparatus for controlling devices connected to home network

• US 7,957,974 B2
• Filed: 03/17/2003
• Issued: 06/07/2011
• Est. Priority Date: 03/15/2002
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Jay-woo, Park, Jun-ho, Cho, Jeong-mi, Hong, Young-jin
• Primary Examiner(s): YEN, ERIC L
• Application Number:
  US10/388,749
• Publication Number:
  US 20030187659A1
• Time in Patent Office: 3,004 Days
• Field of Search: 704270-275, 704/9
• US Class Current: 704/275
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: H04L 12/2803 Home automation networks
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204

Abstract

A method and apparatus for controlling home electronic devices connected to a home network are provided. The method for controlling home electronic devices connected to a home network includes receiving a user voice command and converting the user voice command into a character command; extracting actions and objects from the character command and converting the character command into a logical command; extracting an action list containing a series of actions from the logical command by referring to an action library storing action data for controlling home electronic devices connected to the home network; and converting the series of actions included in the action list into a control signal and controlling the home electronic devices connected to the home network. According to the method and apparatus, user commands to home electronic devices connected to a complicated home network can be simplified such that home electronic devices are controlled conveniently and efficiently.

Claims

1. A method for controlling home electronic devices connected to a home network comprising:

(a) receiving a user voice command and converting the user voice command into a character command;

(b) converting the character command into a logical command, comprising;

(b1) extracting actions and objects from the character command,

(b2) storing semantic information related to the meaning of the extracted actions and semantic information related to the meaning of the extracted objects, and

(b3) analyzing the stored semantic information to remove semantic ambiguity from the actions and objects based on the semantic information of the character command and outputting the logical command based on the analysis, wherein a time that the character command is received is used to interpret a meaning of an object;

(c) generating an action list comprising a series of actions used to execute the logical command by searching an action library storing action data for controlling the home electronic devices connected to the home network, wherein the generated action list is influenced by which electronic devices are currently connected to the home network and by state information of the electronic devices currently connected to the home network; and

(d) converting the series of actions included in the action list into a control signal and controlling the home electronic devices connected to the home network according to the control signal.

2. The method of claim 1, further comprising:

(e) reporting the result of execution of an action controlling a home electronic device connected to the home network.

3. The method of claim 2, wherein the step (e) comprises:

receiving the result of execution of the action from the home electronic device connected to the home network; and

outputting a result message containing the received result of execution of the action.

4. The method of claim 3, wherein the result message includes an execution result success message or an execution result failure message and the execution result failure message includes information on the home electronic device in which an error occurred.

5. The method of claim 1, further comprising:

(f) monitoring the states of the home electronic devices, wherein a state of a home electronic device changes according to the control of the home electronic device connected to the home network.

6. The method of claim 5, wherein the step (f) comprises:

receiving changed home electronic device state information from the home electronic devices connected to the home network; and

updating a device state library which stores the state information of the home electronic devices connected to the home network based on the received home electronic device state information.

7. The method of claim 1, wherein a data structure of the action data includes an action header, a prerequisite field, a sub-action field, and an action effect field.

8. The method of claim 7, wherein the action header field indicates an action, the prerequisite field specifies any prerequisites needed to perform the action, the sub-action field specifies any sub-actions needed to complete execution of the action, and the action effect field specifies a result of execution of the action.

9. The method of claim 8, wherein the step (c) comprises:

(c1) determining whether or not a prerequisite included in the action data corresponding to the logical command is satisfied;

(c2) if the determination result indicates that the prerequisite is not satisfied and an action capable of satisfying the prerequisite in the action library, searching for the action;

(c3) if the determination result indicates that the prerequisite is not satisfied and an action capable of satisfying the prerequisite is not in the action library, outputting an error message;

(c4) if the prerequisite included in the action data corresponding to the logical command is satisfied and there are sub-actions, searching for the sub-actions from the action library; and

(c5) if the prerequisite included in the action data corresponding to the logical command is satisfied and there are no more sub-actions, writing the action in the action list.

10. The method of claim 9, wherein in the step (c1), whether or not the prerequisite is satisfied is determined by referring to a device state library which stores the state information of home electronic devices connected to the home network.

11. The method of claim 10, wherein a data structure of device state data stored in the device state library includes a device ID field which identifies a device, an action field which specifies an action associated with the device, and a device state field which specifies a state of the device with respect to the specified action.

12. The method of claim 1, wherein information stored about the objects is used to interpret a meaning of an action.

31. A non-transitory recording medium on which a method of controlling home electronic devices connected to a home network of claim 1 is recorded using program codes which can be read out and operated by a computer.

32. A non-transitory recording medium on which a method of controlling home electronic devices connected to a home network of claim 2 is recorded using program codes which can be read out and operated by a computer.

33. A non-transitory recording medium on which a method of controlling home electronic devices connected to a home network of claim 5 is recorded using program codes which can be read out and operated by a computer.

13. An apparatus for controlling home electronic devices connected to a home network comprising:

an action library which stores action data for controlling the home electronic devices connected to the home network;

a voice recognition unit which receives a user voice command and recognizes the user voice command as a character command;

a command interpretation unit which converts the character command into a logical command, wherein the command interpretation unit comprises;

a command surface analysis unit which extracts actions and objects from the character command and stores semantic information related to the meaning of the extracted actions and semantic information related to the meaning of the extracted objects, and

a command deep analysis unit which analyzes the stored semantic information to remove semantic ambiguity from the actions and objects based on the semantic information of the character command and outputs the logical command based on the analysis, wherein a time that the character command is received is used to interpret a meaning of an object;

an action planning unit which generates an action list comprising a series of actions used to execute the logical command by searching the action library, wherein the generated action list is influenced by which electronic devices are currently connected to the home network and by state information of the electronic devices currently connected to the home network; and

a home electronic device control unit which converts the series of actions contained in the action list into a control signal and controls the home electronic devices connected to the home network according to the control signal.

14. The apparatus of claim 13, further comprising:

a response management unit which reports the result of execution of an action controlling a home electronic device connected to the home network.

15. The apparatus of claim 14, wherein the response management unit receives the result of execution of the action through a device control unit from the home electronic device connected to the home network and outputs a result message containing the received result of execution of the action.

16. The apparatus of claim 15, wherein the result message is output in the form of voice or characters.

17. The apparatus of claim 15, wherein the result message includes an execution result success message or an execution result failure message and the execution result failure message includes information on the home electronic device in which an error occurred.

18. The apparatus of claim 13, further comprising:

a device monitoring unit which monitors states of home electronic devices which change according to control of the home electronic devices connected to the home network.

19. The apparatus of claim 18, wherein the device monitoring unit receives changed home electronic device state information from the home electronic devices connected to the home network and updates a device state library which stores the state information of home electronic devices connected to the home network based on the received home electronic device state information.

20. The apparatus of claim 13, wherein a data structure of the action data includes an action header, a prerequisite field, a sub-action field, and an action effect field.

21. The apparatus of claim 20, wherein the action header field indicates an action, the prerequisite field specifies any prerequisites needed to perform the action, the sub-action field specifies any sub-actions which are needed to complete execution of the action, and the action effect field specifies a result of execution of the action.

22. The apparatus of claim 21, wherein the action planning unit determines whether or not a prerequisite included in the action data corresponding to the logical command is satisfied, and wherein if the result of determining whether or not a prerequisite included in the action data corresponding to the logical command is satisfied indicates that the prerequisite is not satisfied and an action capable of satisfying the prerequisite is not in the action library, the action planning unit outputs an error message.

23. The apparatus of claim 22, wherein the action planning unit determines whether or not the prerequisite included in the action data corresponding to the logical command is satisfied by referring to a device state library which stores the state information of the home electronic devices connected to the home network.

24. The apparatus of claim 23, wherein a data structure of device state data stored in the device state library includes a device ID field which identifies a device, an action field which specifies an action associated with the device, and a device state field which specifies a state of the device with respect to the specified action.

25. The apparatus of claim 21, wherein the action planning unit determines whether or not a prerequisite included in the action data corresponding to the logical command is satisfied, and wherein if the result of determining whether or not a prerequisite included in the action data corresponding to the logical command is satisfied indicates that the prerequisite is not satisfied and an action capable of satisfying the prerequisite is in the action library, the action planning unit searches for the action in the action library.

26. The apparatus of claim 25, wherein the action planning unit determines whether or not the prerequisite included in the action data corresponding to the logical command is satisfied by referring to a device state library which stores the state information of the home electronic devices connected to the home network.

27. The apparatus of claim 26, wherein a data structure of device state data stored in the device state library stores includes a device ID field which identifies a device, an action which specifies an action associated with the device, and a device state field which specifies a state of the device with respect to the specified action.

28. The apparatus of claim 21, wherein the action planning unit determines whether or not a prerequisite included in the action data corresponding to the logical command is satisfied, and wherein the action planning unit searches for a sub-action in the action library in the event that the prerequisite included in the action data corresponding to the logical command is satisfied and the sub-action is in the action library.

29. The apparatus of claim 21, wherein the action planning unit determines whether or not a prerequisite included in the action data corresponding to the logical command is satisfied, and wherein the action is written in the action list in the event that the prerequisite included in the action data corresponding to the logical command is satisfied and a sub-action is not in the action library.

30. The apparatus of claim 13, wherein information stored about the objects is used to interpret a meaning of an action.