MOBILE TERMINAL AND CONTROLLING METHOD THEREOF

• US 20140164928A1
• Filed: 10/09/2013
• Published: 06/12/2014
• Est. Priority Date: 12/06/2012
• Status: Abandoned Application

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: KIM, Seunghee
• Application Number:
  US14/050,261
• Publication Number:
  US 20140164928A1
• Time in Patent Office: Days
• US Class Current: 715/728
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

A mobile terminal and controlling method thereof are disclosed, which facilitates the control of the mobile terminal using user'"'"'s eyes and gesture and which minimizes user'"'"'s touch inputs for manipulating the mobile terminal. The present invention includes a camera, a microphone, a display unit, and a controller determining a location faced by eyes of a user on the display unit, the controller, if determining an object displayed on the determined location, performing a function corresponding to at least one of a voice recognized via the microphone and a gesture taken via the camera on the object.

Claims

1. A mobile terminal comprising:

a camera configured to recognize a gesture;

a microphone configured to recognize a voice;

a display unit configured to display a first object; and

a controller configured to determine a first location on the display unit that is faced by a user'"'"'s eyes and to perform a first function on the first object displayed at the first location in response to at least the voice recognized via the microphone or the gesture recognized via the camera.

2. The mobile terminal of claim 1, wherein the controller is further configured to control the display unit to display the first object in a visually identifiable manner during a time that the user'"'"'s eyes are maintained on the first object.

3. The mobile terminal of claim 1, wherein:

a plurality of objects are displayed at the first location; and

the controller is further configured to control the display unit to display the plurality of objects such that they are visually identifiable from each other.

4. The mobile terminal of claim 3, wherein the controller is further configured to select one of the plurality of objects as the first object in response to the voice recognized via the microphone.

5. The mobile terminal of claim 1, wherein the controller is further configured to control the display unit to display an indicator corresponding to the first function, the indicator indicating that the first function is performed on the first object.

6. The mobile terminal of claim 1, further comprising:

a memory configured to store information,

wherein the controller is further configured to;

perform a second function on the first object in response to at least a new voice recognized via the microphone or a new gesture recognized via the camera; and

control the memory to store a history of functions and corresponding voice and gesture inputs such that they are related to each other.

7. The mobile terminal of claim 6, wherein the controller is further configured to control the display unit to display the stored history of functions and corresponding voice and gesture inputs.

8. The mobile terminal of claim 1, wherein the controller is further configured to:

determine a second location on the display unit that is faced by the user'"'"'s eyes, wherein a second object is displayed at the second determined location; and

perform a second function on both the first object and the second object.

9. A method of controlling a mobile terminal, the method comprising:

determining a first location on a display unit that is faced by a user'"'"'s eyes;

determining a first object displayed at the first location; and

performing a first function on the first object in response to at least a recognized voice input or a recognized gesture input.

10. The method of claim 9, further comprising:

displaying the first object in a visually identifiable manner during a time that the user'"'"'s eyes are maintained on the first object.

11. The method of claim 9, further comprising:

displaying a plurality of objects at the first location, wherein the plurality of objects are visually identifiable from each other.

12. The method of claim 11, further comprising:

receiving an input via the microphone to select one of the plurality of objects as the first object.

13. The method of claim 9, further comprising:

displaying an indicator on the display unit that indicates that the first function is performed on the first object.

14. The method of claim 9 further comprising:

recognizing at least a new voice input or a new gesture input;

performing a second function corresponding to the at least new voice input or the new gesture input; and

storing the second function and either the at least new voice input or the new gesture input corresponding to the second function such that they are related to each other.

15. The method of claim 14, further comprising:

displaying a history of stored functions and corresponding voice and gesture inputs.

16. The method of claim 9, further comprising:

determining a second location on the display unit faced by the user'"'"'s eyes;

determining a second object corresponding to the second location; and

performing a second function on the first object and the second object.

17. A recording medium comprising commands, the recording medium comprising:

a first command to determine a first location faced by a user'"'"'s eyes on a display unit;

a second command to determine a first object displayed on the display unit at the first location; and

a third command to perform a first function on the first object in response to at least a recognized voice or a recognized gesture.

18. The recording medium of claim 17, further comprising:

a fourth command to display the first object in a visually identifiable manner during the time that the user'"'"'s eyes are maintained on the first object.

19. The recording medium of claim 17, further comprising:

a fourth command to display a plurality of objects at the first location, wherein the plurality of objects are visually identifiable from each other.

20. The recording medium of claim 17, further comprising:

a fourth command to display an indicator on the display unit that indicates that the first function is performed on the first object.