CONTROL METHOD USING VOICE AND GESTURE IN MULTIMEDIA DEVICE AND MULTIMEDIA DEVICE THEREOF

• US 20120127072A1
• Filed: 11/22/2011
• Published: 05/24/2012
• Est. Priority Date: 11/22/2010
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: KIM, Hyeran
• Granted Patent: US 9,390,714 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

A multimedia device and a method for controlling the same are disclosed, in which voice and gesture of a user are recognized by the multimedia device to allow the user to execute a desired operation. The method Includes enabling an input of a remote controller input of a gesture and a voice; receiving user the gesture and the voice through the remote controller; identifying a first command associated with the received gesture; identifying a second command associated with the received voice; comparing the first command and the second command to each other; and performing a function associated with the first or second command when the comparing step indicates that the first command corresponds to the second command. The multimedia device executes the operation desired by the user.

Claims

1. A method for controlling a multimedia device, the method comprising:

enabling input of a gesture and a voice;

receiving the gesture and the voice;

identifying a first command associated with the received gesture;

identifying a second command associated with the received voice;

comparing the first command and the second command to each other; and

performing a function associated with the first or second command when the comparing step indicates that the first command corresponds to the second command.

2. The method according to claim 1, wherein the identifying of the first command includes searching for the function, which can be executed in the multimedia device, from a database based on feature information of the gesture, and determining whether gesture information matches the feature information of the gesture;

and

the identifying of the second command includes searching for the function, which can be executed in the multimedia device, from the database based on feature information of the voice, and determining whether voice information matches the feature information of the voice.

3. The method according to claim 1, wherein the multimedia device performs the identifying of the first and second commands and the comparing step after receiving the gesture and the voice.

4. The method according to claim 1, further comprising a remote controller associated with the multimedia device, wherein the remote controller performs the identifying of the first and second commands and the comparing step after receiving the gesture and the voice.

5. The method according to claim 4, wherein the enabling input of the gesture and the voice includes:

communicating specific data for enabling the input of the gesture and the voice between the multimedia device and the remote controller through wireless communication; and

enabling the remote controller if the specific data is communicated for a certain time or more.

6. The method according to claim 4, wherein the receiving of the gesture and the voice includes:

receiving gesture information through a gyro sensor of the remote controller; and

receiving voice information through a microphone of the remote controller.

7. The method according to claim 3, wherein the receiving of the gesture and the voice includes:

receiving gesture information through a camera of the multimedia device; and

receiving voice information through a microphone of the multimedia device.

8. The method according to claim 4, wherein the remote controller is one that can be attached to a hand of a user, and the gesture information includes direction information and acceleration information based on motion of the hand.

9. The method according to claim 2, wherein the determining of whether the gesture information matches the feature information of the gesture includes:

extracting direction information of the gesture; and

determining that the function is associated with the extracted direction information from the database, and

the determining of whether the voice information matches the feature information of the voice includes;

extracting waveform information of the voice; and

determining that the function is associated with the extracted waveform information.

10. The method according to claim 1, wherein the performing of the function includes:

outputting a warning message if any loss occurs in an operation performed by the multimedia device after the function is executed; and

executing the function if a confirmation command for the warning message is received.

11. A multimedia device that can be controlled by a remote controller, the multimedia device comprising:

a user interface module configured to receive a gesture and a voice;

a memory module configured to store a first command and a second command; and

a control module configured to identify the first command associated with the received gesture, identify the second command associated with the received voice, compare the first command and the second command to each other, and perform a function associated with the first or second command when the comparison indicates that the first command corresponds to the second command.

12. The multimedia device according to claim 11, wherein the control module identifies the first command by searching for the function, which can be executed in the multimedia device based on feature information of the gesture, and determines whether gesture information matches the feature information of the gesture, and

wherein the control module identifies the second command by searching for the function, which can be executed in the multimedia device based on feature information of the voice, and determines whether voice information matches the feature information of the voice.

13. The multimedia device according to claim 11, wherein the control module performs the identification of the first and second commands, and the comparison after receiving the gesture and the voice.

14. The multimedia device according to claim 11, further comprising a remote controller associated with the multimedia device, wherein the remote controller receives input of the gesture and the voice.

15. The multimedia device according to claim 14, wherein the remote control is enabled for input of the gesture and the voice by communication of specific data for enabling the input of the gesture and the voice between the multimedia device and the remote controller through wireless communication for a certain time or more.

16. The multimedia device according to claim 14, wherein the remote controller receives input of the gesture by receiving gesture information through a gyro sensor of the remote controller, and receives input of the voice by receiving voice information through a microphone of the remote controller.

17. The multimedia device according to claim 16, wherein the remote controller is one that can be attached to a hand of a user, and the gesture information includes direction information and acceleration information based on motion of the hand.

18. The multimedia device according to claim 13, wherein the multimedia device receives the gesture through a camera of the multimedia device, and receives the voice information through a microphone of the multimedia device.

19. The method according to claim 12, wherein the determination of whether the gesture information matches the feature information of the gesture includes:

extracting direction information of the gesture; and

determining that the function is associated with the extracted direction information, and

the determination of whether the voice information matches the feature information of the voice includes;

extracting waveform information of the voice; and

determining that the function is associated with the extracted waveform information.

20. A remote controller that can control a multimedia device, the remote controller comprising:

a wireless communication module configured to transmit and receiving data to and from the multimedia device;

a sensing module configured to receive a gesture and a voice;

a memory module configured to store a first command and a second command; and

a control module configured to identify the first command associated with the received gesture, identify the second command associated with the received voice, compare the first command and the second command to each other, and perform a function associated with the first or second command when the comparison indicates that the first command corresponds to the second command.