Method for controlling electronic apparatus based on voice recognition and motion recognition, and electronic apparatus applying the same

• US 9,002,714 B2
• Filed: 08/06/2012
• Issued: 04/07/2015
• Est. Priority Date: 08/05/2011
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Jung-geun, Kim, Yoo-tai, Yu, Seung-dong, Han, Sang-jin, Ryu, Hee-seob
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US13/567,354
• Publication Number:
  US 20130033643A1
• Time in Patent Office: 974 Days
• Field of Search: 704/270, 704/275
• US Class Current: 704/275
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04895 Guidance during keyboard input operation, e.g. prompting
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G08C 2201/31 Voice input
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/439 Processing of audio elementary streams
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4821 using a grid, e.g. sorted out by channel and broadcast time

Abstract

A method for controlling an electronic apparatus which uses voice recognition and motion recognition, and an electronic apparatus applying the same are provided. In a voice task mode, in which voice tasks are performed according to recognized voice commands, the electronic apparatus displays voice assistance information to assist in performing the voice tasks. In a motion task mode, in which motion tasks are performed according to recognized motion gestures, the electronic apparatus displays motion assistance information to aid in performing the motion tasks.

Claims

1. A method for controlling an electronic apparatus, the method comprising:

displaying, by using a processor, voice assistance information comprising a plurality of voice items on a screen of the electronic apparatus, each of the plurality of voice items corresponding to a voice task in a voice task mode;

recognizing, by using a processor, a first voice command corresponding to a first voice item from among the plurality of voice items;

performing, by using a processor, a first voice task corresponding to the first voice item;

displaying, by using a processor, motion assistance information comprising a plurality of simultaneously displayed motion items on a screen of the electronic apparatus, each of the plurality of simultaneously displayed motion items corresponding to a motion task in a motion task mode;

recognizing, by using a processor, a first motion gesture corresponding to a first motion item from among the plurality of simultaneously displayed motion items; and

performing, by using a processor, a first motion task corresponding to the first motion item,

wherein the performing the first voice task comprises displaying additional item information regarding the first voice item,

wherein the performing the first voice task further comprises recognizing a second voice command corresponding to the additional item information, and

wherein each of the plurality of simultaneously displayed motion items includes a guide image which represents a user motion to perform a task corresponding to a motion item.

2. The method as claimed in claim 1, wherein the recognizing the first voice command comprises providing at least one of among visual feedback and auditory feedback indicating at least one of among the first voice item and the first voice task.

3. The method as claimed in claim 1, wherein the recognizing the first motion gesture comprises providing at least one of among visual feedback and auditory feedback indicating at least one of among the first motion item and the first motion task.

4. The method as claimed in claim 1, wherein the performing the first voice task comprises performing a task corresponding to both the first voice item and the recognized second voice command.

5. The method as claimed in claim 1, wherein the additional item information comprises at least one of among an additional command, a guide message, and an utterance example regarding the first voice item.

6. The method as claimed in claim 1, wherein the additional item information comprises at least one of among an image and a text which indicates a designation of one of among the voice task mode and the motion task mode.

7. The method as claimed in claim 1, wherein, if the first voice task comprises a channel shortcut, the additional item information comprises at least one of among a guide message and an utterance example regarding utterance of at least one of among an identifier of a channel, a program name, and a channel name.

8. The method as claimed in claim 7, further comprising:

recognizing, by using a processor, a second voice command corresponding to one of among an identifier of a channel name, a program name, and a channel name;

performing, by using a processor, a channel shortcut to reproduce a channel corresponding to the one of among the identifier of the channel, the program name and the channel name.

9. The method as claimed in claim 1, wherein, if the first voice comprises a power off, the additional item information comprises at least one of among a guide message asking a question as to whether to perform power off and an utterance example regarding an answer to the question.

10. The method as claimed in claim 1, wherein each of the plurality of voice items are utterable commands.

11. The method as claimed in claim 1, wherein each of the plurality of simultaneously displayed motion items further comprises a text which indicates a motion task corresponding the recognizable motion gesture.

12. The method as claimed in claim 1, wherein the voice assistance information comprises at least one of among an image and a text which indicates a designation of the voice task mode, and

the motion assistance information comprises at least one of among an image and a text which indicates a designation of the motion task mode.

13. The method as claimed in claim 1, wherein the voice task mode is changed to the motion task mode in response to a motion start gesture,

wherein the motion task mode is changed to the voice task mode in response to a voice start command,

wherein the motion start gesture comprises a pre-set motion gesture to enter the motion task mode, and

wherein the voice start command comprises a pre-set voice command to enter the voice task mode.

14. The method as claimed in claim 1, wherein, if the first voice command comprises at least one of among a channel name, a program name, and a channel number, the first voice task comprises a channel shortcut to reproduce a channel corresponding to the one of among the channel name, the program name and the channel number.

15. The method as claimed in claim 1, wherein:

if the first motion gesture comprises a channel-up gesture, the first motion task comprises changing a current channel to a next channel, and

if the first motion gesture comprises a channel-down gesture, the first motion task comprises changing the current channel to a previous channel.

16. The method as claimed in claim 1, wherein:

if the first voice command comprises a volume level, the first task comprises changing a current volume level setting to a volume level setting corresponding to the first voice command,

if the first motion gesture comprises a volume-up gesture, the first motion task comprises increasing the current volume level by a predetermined amount, and

if the first motion gesture comprises a volume-down gesture, the first motion task comprises reducing the current volume by the predetermined amount.

17. The method as claimed in claim 1, if the first voice item comprises mute, the first voice task comprises performing the mute and displaying voice assistance information comprising a second voice item corresponding to unmute.

18. The method as claimed in claim 1, wherein the voice assistance information further comprises a recognition candidate comprising at least one of among a phrase similar to the first voice command, a voice command which is similar to the first voice command, a task which corresponds to the similar voice command, a motion gesture which is similar to the first motion gesture, and a motion task which corresponds to the similar motion gesture.

19. The method as claimed in claim 18, further comprising:

displaying, by using a processor, the voice assistance information including the recognition candidate;

changing, by using a processor, from the voice task mode to the motion task mode in response to a motion start command;

continuing to display, by using a processor, the recognition candidate and displaying the motion assistance information.

20. The method as claimed in claim 19, wherein the continuing to display the recognition candidate comprises displaying a focus on the recognition candidate.

21. The method as claimed in claim 1, wherein the motion assistance information further comprises a recognition candidate comprising at least one of among a phrase similar to the first voice command, a voice command which is similar to the first voice command, a voice task which corresponds to the similar voice command, a motion gesture which is similar to the first motion gesture, and a motion task which corresponds to the similar motion gesture.

22. The method as claimed in claim 21, further comprising:

displaying, by using a processor, the motion assistance information including the recognition candidate;

changing, by using a processor, from the motion task mode to the voice task mode;

continuing to display, by using a processor, the recognition candidate

and displaying, by using a processor, the voice assistance information.

23. The method as claimed in claim 22, wherein the continuing to display the recognition candidate comprises displaying an index comprising at least one of a figure and a letter corresponding to the recognition candidate.

24. The method as claimed in claim 1, wherein the displaying additional item information comprises replacing the plurality of voice items.

25. The method as claimed in claim 24, wherein the displaying additional item information further comprises omitting driving of a timer to stop the voice task mode.

26. An electronic apparatus comprising:

a display unit which displays voice assistance information and additional item information on a screen of the electronic apparatus;

a controller which;

controls the display unit to display voice assistance information comprising a plurality of voice items on a screen of the electronic apparatus;

recognizes a first voice command corresponding to a first voice item from among the plurality of voice items;

controls the display unit to display additional item information regarding the first voice item on a screen of the electronic apparatus;

recognizes a second voice command corresponding to the additional item information;

performs a first voice task corresponding to the first voice item and the second voice command; and

displays motion assistance information comprising a plurality of simultaneously displayed motion items on a screen of the electronic apparatus, each of the plurality of simultaneously displayed motion items corresponding to a motion task in a motion task mode,

wherein each of the plurality of simultaneously displayed motion items includes a guide image which represents a user motion to perform a task corresponding to a motion item.

27. A method for controlling an electronic apparatus, the method comprising:

in a voice task mode in which a plurality of voice tasks are performed, each of the plurality of voice tasks in response to a voice command, displaying, by using a processor, voice assistance information comprising a plurality of items on a screen of the electronic apparatus;

recognizing, by using a processor, a first voice command corresponding to a first voice item from among the plurality of voice items;

displaying, by using a processor, additional item information regarding the first voice item on a screen of the electronic apparatus;

recognizing, by using a processor, a second voice command corresponding to the additional item information;

performing, by using a processor, a first voice task corresponding to the first voice item and the second voice command; and

in a motion task mode in which a plurality of motion tasks are performed, displaying, motion assistance information comprising a plurality of simultaneously displayed motion items on a screen of the electronic apparatus, each of the plurality of simultaneously displayed motion items corresponding to a motion task in a motion task mode,

wherein each of the plurality of simultaneously displayed motion items includes a guide image which represents a user motion to perform a task corresponding to a motion item.

28. An electronic apparatus comprising:

a display which;

in a voice task mode, displays voice assistance information comprising a plurality of voice items on a screen of the electronic apparatus, each of the plurality of voice items corresponding to a voice task; and

in a motion task mode, displays motion assistance information comprising a plurality of simultaneously displayed motion items on a screen of the electronic apparatus, each of the plurality of simultaneously displayed motion items corresponding to a motion task; and

a controller which;

recognizes a first voice command corresponding to a first voice item from among the plurality of voice items, and performs a first voice task corresponding to the first voice item,

recognizes a first motion gesture corresponding to a first motion item from among the plurality of simultaneously displayed motion items, and performs a first motion task corresponding to the first motion item,

wherein the controller controls the display to display additional item information regarding the first voice item,

wherein the controller recognizes a second voice command corresponding to the additional item information, and

wherein each of the plurality of simultaneously displayed motion items includes a guide image which represents a user motion to perform a task corresponding to a motion item.

29. The electronic apparatus as claimed in claim 28, wherein each of the plurality of simultaneously displayed motion items further comprises a text which indicates a motion task corresponding the recognizable motion gesture.

30. The electronic apparatus as claimed in claim 28, wherein the voice assistance information comprises at least one of among an image and a text which indicates a designation of the voice task mode, and

the motion assistance information comprises at least one of among an image and a text which indicates a designation of the motion task mode.