MOBILE TERMINAL AND CONTROL METHOD THEREOF

• US 20150089440A1
• Filed: 09/22/2014
• Published: 03/26/2015
• Est. Priority Date: 09/24/2013
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Choi, Hyonmi, Kim, Hyosung
• Granted Patent: US 9,753,632 B2
• Time in Patent Office: Days
• US Class Current: 715/784
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04855 Interaction with scrollbars
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

Disclosed are a mobile terminal and a control method thereof. The mobile terminal includes a sensing unit, a display unit and a controller. The sensing unit senses a user'"'"'s sight line. The display unit displays screen information. The controller controls the screen information output on the display unit, based on sight line information corresponding to the sensed user'"'"'s sight line. When a user'"'"'s gesture on the display unit is sensed at a time point corresponding to that when the user'"'"'s sight line is sensed, the controller limits control of the screen information based on the sight line information.

Claims

1. A mobile terminal comprising:

a sensing unit configured to sense a user'"'"'s sight line;

a display unit configured to display screen information; and

a controller configured to control the displayed screen information based on sight line information corresponding to the sensed user'"'"'s sight line,

wherein when a user'"'"'s gesture on the display unit is sensed at a time corresponding to when the user'"'"'s sight line is sensed, the controller to limit control of displaying the screen information based on the sight line information.

2. The mobile terminal of claim 1, wherein when the user'"'"'s gesture on the display unit is sensed, the controller controls the screen information based on the sight line information while a sight line control mode is activated for controlling the screen information corresponding to the user'"'"'s sight line.

3. The mobile terminal of claim 2, wherein when a control command at the display unit is not received for a predetermined time after the user'"'"'s gesture is sensed, the displayed screen information is controlled based on the sight line information.

4. The mobile terminal of claim 3, wherein the user'"'"'s gesture is a user'"'"'s movement with respect to the display unit, and

wherein the control command is received based on a touch input to the display unit.

5. The mobile terminal of claim 3, wherein the sight line information is one of a moving speed of the sight line, a moving direction of the sight line and a moving trace of the sight line.

6. The mobile terminal of claim 1, wherein the sensing unit includes a camera sensor, and

wherein the sight line information is obtained from eye images of the user, wherein the eye images are received through the camera sensor.

7. The mobile terminal of claim 1, wherein the controller changes at least one portion of the displayed screen information based on the sight line information.

8. The mobile terminal of claim 1, wherein a screen is scrolled based on sight line information corresponding to a moving trace of the sight line,

wherein a currently output page is changed to another page based on sight line information corresponding to a moving direction of the sight line.

9. The mobile terminal of claim 1, wherein the user'"'"'s gesture corresponds to movement of a user'"'"'s finger with respect to the display unit, and

wherein the controller determines a relative distance between the display unit and the user'"'"'s finger, and the controller determines whether to control the screen information based on the sight line information according to the determined relative distance.

10. The mobile terminal of claim 9, wherein when the determined relative distance is less than a predetermined reference distance, the controller does not control the screen information based on the sight line information, and

wherein when the determined relative distance is greater than the predetermined reference distance, the controller controls the screen information based on the sight line information.

11. The mobile terminal of claim 1, wherein when there exist data related to the displayed screen information and the data is not currently displayed on the display unit, the controller controls the sensing unit to sense the sight line such that the data not currently displayed on the display unit is to be output, corresponding to the user'"'"'s sight line.

12. The mobile terminal of claim 1, wherein when the user'"'"'s gesture is sensed at a same time when the sight line information corresponding to the user'"'"'s sight line is sensed, the controller controls the screen information according to one of the sight line information and the user'"'"'s gesture, based on a predetermined priority order.

13. The mobile terminal of claim 12, wherein the predetermined priority order changes based on a type of the displayed screen information.

14. The mobile terminal of claim 13, wherein when the screen information is a web browser screen, the predetermined priority order is to control the screen information based on the sight line information, and

wherein when the screen information is a keyboard input screen, the predetermined priority order is to control the screen information based on the user'"'"'s gesture.

15. The mobile terminal of claim 1, wherein the controller analyzes a direction of the user'"'"'s sight line, by using the sight line information, and the controller rotates the displayed screen information to correspond to the direction of the user'"'"'s sight line.

16. The mobile terminal of claim 15, further comprising a main body,

wherein rotation of the main body is sensed during activation of an automatic rotation mode for automatically rotating the displayed screen information according to the rotation of the main body, the controller analyzes the direction of the user'"'"'s sight line, and limits the rotation of the displayed screen information when the direction of the user'"'"'s sight line corresponds to the display direction of the displayed screen information.

17. The mobile terminal of claim 1, wherein when the user'"'"'s sight line is not sensed for a predetermined time, the controller changes the mobile terminal to a saving mode for reducing current consumption when the user does not use the mobile terminal.

18. A method for controlling a mobile terminal, the method comprising:

outputting screen information at a display unit;

sensing a user'"'"'s sight line; and

controlling the displayed screen information, based on sight line information corresponding to the sensed user'"'"'s sight line, wherein when a user'"'"'s gesture on the display unit is sensed at a time corresponding to when the user'"'"'s sight line is sensed, limiting control of displaying the screen information based on the sight line information.

19. The method of claim 18, wherein when the user'"'"'s gesture on the display unit is sensed, the screen information is controlled based on the sight line information while a sight line control mode is activated for controlling the screen information corresponding to the user'"'"'s sight line.

20. The method of claim 19, wherein when a control command through the display unit is not received for a predetermined time after sensing the user'"'"'s gesture, the displayed screen information is controlled based on the sight line information.

21. The method of claim 20, wherein the user'"'"'s gesture is a user'"'"'s movement with respect to the display unit, and

wherein the control command is received based on a touch input to the display unit.

22. The method of claim 20, wherein the sight line information is one of a moving speed of the sight line, a moving direction of the sight line and a moving trace of the sight line.

23. The method of claim 18, further comprising obtaining, from a camera sensor, eye images of the user, and obtaining sight line information based on the eye images.

24. The method of claim 18, further comprising changing at least one portion of the displayed screen information based on the sight line information.

25. The method of claim 18, wherein when the user'"'"'s gesture is sensed at a same time when the sight line information corresponding to the user'"'"'s sight line is sensed, the screen information is controlled according to one of the sight line information and the user'"'"'s gesture, based on a predetermined priority order.