Mobile terminal and control method thereof

• US 9,753,632 B2
• Filed: 09/22/2014
• Issued: 09/05/2017
• Est. Priority Date: 09/24/2013
• Status: Expired due to Fees

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Choi, Hyonmi, Kim, Hyosung
• Primary Examiner(s): Kujundzic, Dino
• Application Number:
  US14/492,848
• Publication Number:
  US 20150089440A1
• Time in Patent Office: 1,079 Days
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04855 Interaction with scrollbars
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

Disclosed are a mobile terminal and a control method thereof. The mobile terminal includes a sensing unit, a display unit and a controller. The sensing unit senses a user'"'"'s sight line. The display unit displays screen information. The controller controls the screen information output on the display unit, based on sight line information corresponding to the sensed user'"'"'s sight line. When a user'"'"'s gesture on the display unit is sensed at a time point corresponding to that when the user'"'"'s sight line is sensed, the controller limits control of the screen information based on the sight line information.

Claims

1. A mobile terminal comprising:

a sensing unit configured to sense a user'"'"'s sight line;

a touch screen; and

a controller configured to;

display, on the touch screen, screen information;

receive a first control command using sight line information corresponding the user'"'"'s sight line and a second control command using a user'"'"'s gesture applied to the touch screen;

determine a priority order based on a type of the screen information when the first control command and the second control command is received simultaneously, wherein when the screen information is a web browser page the first control command is to have higher priority than the second control command, wherein when the screen information is a keyboard input screen, the second control command is to have higher priority than the first control command;

determine which control command among the first control command and the second control command is to be processed, based on the determined priority order; and

execute a function related to the screen information in response to the determined control command.

2. The mobile terminal of claim 1, wherein the user'"'"'s gesture is a user'"'"'s movement with respect to the touch screen, and

wherein the second control command is received based on a touch input to the touch screen.

3. The mobile terminal of claim 1, wherein the sight line information is one of a moving speed of the sight line, a moving direction of the sight line and a moving trace of the sight line.

4. The mobile terminal of claim 1, wherein the sensing unit includes a camera sensor, and

wherein the sight line information is obtained from eye images of the user, wherein the eye images are received through the camera sensor.

5. The mobile terminal of claim 1, wherein the controller changes at least one portion of the displayed screen information based on the sight line information.

6. The mobile terminal of claim 1, wherein a screen is scrolled based on sight line information corresponding to a moving trace of the sight line,

wherein a currently output page is changed to another page based on sight line information corresponding to a moving direction of the sight line.

7. The mobile terminal of claim 1, wherein the user'"'"'s gesture corresponds to movement of a user'"'"'s finger with respect to the touch screen, and

wherein the controller determines a relative distance between the touch screen and the user'"'"'s finger, and the controller determines whether to control the screen information based on the sight line information according to the determined relative distance.

8. The mobile terminal of claim 1, wherein when there exist data related to the displayed screen information and the data is not currently displayed on the touch screen, the controller controls the sensing unit to sense the sight line such that the data not currently displayed on the touch screen is to be output, corresponding to the user'"'"'s sight line.

9. The mobile terminal of claim 1, wherein the controller analyzes a direction of the user'"'"'s sight line, by using the sight line information, and the controller rotates the displayed screen information to correspond to the direction of the user'"'"'s sight line.

10. The mobile terminal of claim 9, further comprising a main body,

wherein rotation of the main body is sensed during activation of an automatic rotation mode for automatically rotating the displayed screen information according to the rotation of the main body, the controller analyzes the direction of the user'"'"'s sight line, and limits the rotation of the displayed screen information when the direction of the user'"'"'s sight line corresponds to the display direction of the displayed screen information.

11. The mobile terminal of claim 1, wherein when the user'"'"'s sight line is not sensed for a specific predetermined time, the controller changes the mobile terminal to a saving mode for reducing current consumption when the user does not use the mobile terminal.

12. A method for controlling a mobile terminal, the method comprising:

outputting screen information at a touch screen;

sensing a user'"'"'s sight line and a user'"'"'s gesture applied to the touch screen;

receiving a first control command using sight line information corresponding the user'"'"'s sight line and a second control command using a user'"'"'s gesture applied to the touch screen;

determining a priority order based on a type of the screen information when the first control command and the second control command is received simultaneously, wherein when the screen information is a web browser page, the first control command is to have higher priority than the second control command, and wherein when the screen information is a keyboard input screen, the second control command is to have higher priority than the first control command;

determining which control command among the first control command and the second control command is to be processed, based on the determined predetermined priority order, wherein the predetermined priority order is based on a type of the screen information; and

execute a function related to the screen information in response to the determined control command.

13. The method of claim 12, wherein the user'"'"'s gesture is a user'"'"'s movement with respect to the touch screen, and

wherein the control command is received based on a touch input to the touch screen.

14. The method of claim 12, wherein the sight line information is one of a moving speed of the sight line, a moving direction of the sight line and a moving trace of the sight line.

15. The method of claim 12, further comprising obtaining, from a camera sensor, eye images of the user, and obtaining sight line information based on the eye images.

16. The method of claim 12, further comprising changing at least one portion of the displayed screen information based on the sight line information.