Method for operating user functions based on eye tracking and mobile device adapted thereto

• US 9,092,051 B2
• Filed: 11/20/2012
• Issued: 07/28/2015
• Est. Priority Date: 11/29/2011
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Park, Kyungdae, Kang, Jiyoung, Koh, Sanghyuk, Park, Mijung, Oh, Saegee, Lee, Chihoon
• Primary Examiner(s): Bost, Dwayne
• Assistant Examiner(s): Martinez Quiles, Ivelisse
• Application Number:
  US13/681,719
• Publication Number:
  US 20130135196A1
• Time in Patent Office: 980 Days
• US Class Current: 1/1
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

An eye tracking based user function controlling method and a mobile device adapted thereto are provided. A camera unit of a mobile device is activated while a specific user function is executed. A gaze angle of a user'"'"'s eye is acquired from an image obtained via the camera unit. An eye tracking function is executed in which execution state is controlled according to the gaze angle.

Claims

1. A method for operating user functions in a mobile device having a display, comprising:

executing a specific user function;

acquiring an image via a camera unit of the mobile device while the specific user function is executed;

acquiring a gaze angle of a user'"'"'s eye from the acquired image and determining from the acquired image whether the user gazes at the display; and

executing an eye tracking function in which execution state of the user function is controlled according to the gaze angle, wherein a page turning interval speed is controlled according to a time period for which a gaze angle at which the user gazes at the same point on the display is acquired.

2. The method of claim 1, further comprising:

prior to activating the camera unit, determining whether the specific user function is associated with eye tracking, and if so, whether an eye tracking mode is activated.

3. The method of claim 1, wherein the execution of an eye tracking function further comprises at least one of the following:

reducing a level of illumination of the display when the acquired gaze angle is a first gaze angle indicating that the user doesn'"'"'t gaze at the display;

turning off the display;

terminating the user function execution;

pausing the user function execution; and

setting or unlocking a touch lock of a touch panel of the display.

4. The method of claim 3, wherein the execution of an eye tracking function further comprises at least one of the following:

if the acquired gaze angle is a second gaze angle indicating that the user gazes at the display, restoring an illumination level of the display to an original level of illumination;

turning on the display;

re-executing the user function;

resuming the user function execution that is paused; and

unlocking or setting a touch lock of the touch panel that is locked.

5. The method of claim 1, further comprising, when the specific user function is a content playback function, pausing or terminating execution of the user function comprises pausing content playback when determining that the user does not gaze at the display over a preset period of time, and, following the pausing, resuming content playback when determining that the user again gazes at the display.

6. The method of claim 1, wherein the execution of an eye tracking function comprises:

maintaining a current screen of the display when the acquired gaze angle is a second gaze angle indicating that the user gazes at the display.

7. The method of claim 1, wherein the acquisition of a gaze angle comprises:

detecting presence of a face of a subject;

automatically focusing on an iris in the detected face; and

acquiring a gaze of the automatically focused iris of the subject.

8. The method of claim 1, wherein the execution of an eye tracking function comprises:

when the acquired gaze angle is a second gaze angle indicating that the user gazes at the display in a state where a call connection request message is received, automatically establishing the call connection.

9. The method of claim 1, further comprising pausing or terminating execution of the user function when determining that the user does not gaze at the display over a preset period of time.

10. A method for operating user functions in a mobile device, comprising:

executing an e-book function;

acquiring an image via a camera unit of the mobile device while the e-book function is executed;

acquiring a gaze angle of a user'"'"'s eye from the acquired image;

turning a page of e-book content when the acquired gaze angle is a gaze angle at which the user gazes at a certain point in a display unit of the mobile device where the e-book content is displayed;

determining an average page turning interval; and

periodically disabling the camera unit for a camera disabling time shorter than, and determined in relation to, the average page turning interval.

11. A mobile device comprising:

a camera unit that acquires images of a subject;

a controller configured to acquire a gaze angle of a user'"'"'s eye from an image acquired via the camera unit, and to execute an eye tracking function in which an execution state of a user function is controlled according to the gaze angle; and

a storage unit that stores data corresponding to the user function, and a program to track gaze angles of a user'"'"'s eye;

wherein the controller automatically establishes a call connection, if the acquired gaze angle is an angle indicating that the user gazes at a display unit of the mobile device in a state where a call connection request message is received.

12. The mobile device of claim 11, wherein the storage unit stores a list of user functions associated with eye tracking.

13. The mobile device of claim 11, wherein if the acquired gaze angle is an angle indicating that the user doesn'"'"'t gaze at the display unit, an illumination level of the display unit is reduced or the display unit is turned off.

14. The mobile device of claim 13, wherein the controller disables the camera unit if the display unit is turned off or the user function execution is terminated.

15. The mobile device of claim 11, wherein, if the controller acquires a first gaze angle indicating that the user doesn'"'"'t gaze at the display unit over a preset period of time, it pauses or terminates the user function execution.

16. The mobile device of claim 11, wherein the controller:

sets or unlocks a touch lock of a touch panel of the display unit if the acquired gaze angle is a first gaze angle indicating that the user doesn'"'"'t gaze at the display unit; and

unlocks or sets a touch lock of a touch panel of the display unit if the acquired gaze angle is a second gaze angle indicating that the user gazes at the display unit.

17. The mobile device of claim 11, wherein the camera unit automatically focuses on the iris of the user'"'"'s eye during the acquisition of images of a subject.

18. A mobile device comprising:

a camera unit that acquires images of a subject;

a controller configured to acquire a gaze angle of a user'"'"'s eye from an image acquired via the camera unit, and to execute an eve tracking function in which an execution state of a user function is controlled according to the gaze angle; and

a storage unit that stores data corresponding to the user function, and a program to track gaze angles of a user'"'"'s eye;

wherein the controller is configured to determine an average page turning interval of e-book content, and periodically disable the camera unit for a camera disabling time shorter than, and determined in relation to, the average page turning interval.

19. The mobile device of claim 18, wherein the controller turns a page of e-book content when the acquired gaze angle is a gaze angle at which the user gazes at a certain point in a display unit of the mobile device where the e-book content is displayed.

20. The mobile device of claim 19, wherein the controller controls a page turning interval speed according to a time period at which a gaze angle at which the user gazes at the same point in the display unit is acquired.