METHOD AND APPARATUS FOR USER INTERFACE USING GAZE INTERACTION

• US 20140232638A1
• Filed: 01/27/2014
• Published: 08/21/2014
• Est. Priority Date: 02/21/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Choi, Chang Kyu, Suh, Sung Joo, Park, Joon Ah, Park, Du Sik
• Granted Patent: US 10,324,524 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

A method and apparatus for a user interface using a gaze interaction is disclosed. The method for the user interface using the gaze interaction may include obtaining an image including eyes of a user, estimating a gaze position of the user, using the image including the eyes of the user, and determining whether to activate a gaze adjustment function for controlling a device by a gaze of the user, based on the gaze position of the user with respect to at least one toggle area on a display.

Claims

1. A method for a user interface using a gaze interaction, the method comprising:

obtaining an image including eyes of a user, using a camera;

estimating a gaze position of the user using the image including the eyes of the user; and

determining whether to activate a gaze adjustment function for controlling a device using a gaze of the user, based on a gaze position of the user with respect to at least one toggle area on a display.

2. The method of claim 1, wherein the obtaining of the image including the eyes of the user comprises:

obtaining an image including the eyes of the user using at least two internal light sources included in the display.

3. The method of claim 1, wherein the at least one toggle area comprises:

at least one calibration point.

4. The method of claim 1, wherein the at least one toggle area comprises:

at least one of a plurality of objects displayed on the display.

5. The method of claim 2, wherein the estimating of the gaze position of the user comprises:

extracting respective positions of at least two glints, being an area in which the at least two internal light sources are displayed on the eyes of the user, respectively, based on the image including the eyes of the user;

calculating a position of a cornea center of the eyes of the user, using respective positions of the at least two internal light sources and the respective positions of the at least two glints;

calculating respective positions of a pupil center of each of the eyes of the user, using the respective positions of the cornea center of the eyes of the user;

calculating respective directions of an optical axis of the eyes of the user, using the respective positions of the cornea center and the respective positions of the pupil center;

calculating respective directions of a visual axis of the eyes of the user, based on the respective directions of the optical axis; and

identifying the gaze position, using an intersection point of respective visual axes of the eyes of the user.

6. The method of claim 2, further comprising:

removing noise by at least one external light source,

wherein the removing of the noise by the at least one external light source comprises;

analyzing an image including the eyes of the user; and

separating the at least two internal light sources and the at least one external light source, using a result of the analysis.

7. The method of claim 6, wherein the removing of the noise by the at least one external light source further comprises:

changing at least one of a position, a color, and a shape of the at least two internal light sources, and

estimating the gaze position of the user, based on the changing of the at least two internal light sources.

8. The method of claim 1, wherein the determining of whether to activate the gaze adjustment function comprises:

determining whether the gaze position matches the at least one toggle area; and

setting the gaze adjustment function to be an activation mode or a deactivation mode, based on a result of the determination.

9. The method of claim 8, wherein the determining of whether the gaze position matches the at least one toggle area further comprises:

determining whether the gaze position matches the at least one toggle area within a predetermined period of time.

10. The method of claim 8, wherein the determining of whether the gaze position matches the at least one toggle area further comprises:

determining whether the gaze position matches the at least one toggle area, based on a predetermined pattern.

11. The method of claim 8, wherein the determining of whether to activate the gaze adjustment function further comprises:

setting a default value of the gaze adjustment function to be the deactivation mode;

setting the gaze adjustment function to be the activation mode during the predetermined period of time when the gaze position matches the at least one toggle area, based on a result of the determination.

12. The method of claim 2, further comprising:

setting at least two vertex areas from among a plurality of vertex areas of the display to be the at least two internal light sources.

13. A method for a user interface using a gaze interaction, the method comprising:

emitting light using at least two internal light sources included in a display;

obtaining, by way of a camera, an image including eyes of a user, wherein reflections of the at least two internal light sources are displayed on the eyes of the user; and

estimating a gaze position of the user, using the image of the eyes of the user.

14. An apparatus for a user interface using a gaze interaction, the system comprising:

a display;

a camera;

an image obtaining unit to obtain an image including eyes of a user, using the camera;

a gaze position estimating unit to estimate the gaze position of the user using the image including the eyes of the user; and

a mode determining unit to determine whether to activate a gaze adjustment function for controlling a device using the gaze of the user, based on a gaze position of the user with respect to at least one toggle area on the display.

15. An apparatus for a user interface using a gaze interaction, the system comprising:

a display;

at least two internal light sources included in the display;

a camera;

an image obtaining unit to obtain, by way of the camera, an image including eyes of a user, wherein reflections of the at least two internal light sources are displayed on the eyes of the user; and

a gaze position estimating unit to estimate the gaze position of the user, using the image including the eyes of the user.

16. A user interface method in a mobile device having a display, the method comprising:

displaying an icon on the display while a user waits for content to be loaded onto the display; and

estimating a gaze position of the user by using the icon as a calibration point.

17. A method of unlocking content of a mobile device having a display with a plurality of toggle areas, the method comprising:

monitoring a gaze position of a user;

determining, based on the monitored gaze position, whether the user has gazed at the toggle areas according to a predetermined pattern; and

unlocking the content of the mobile device when it has been determined that the user has gazed at the toggle areas according to the predetermined pattern.

18. A method of locking content of a mobile device having a display with a plurality of toggle areas, the method comprising:

monitoring a gaze position of a user;

determining, based on the monitored gaze position, whether the user has gazed at the toggle areas according to a predetermined pattern; and

locking the content of the mobile device when it has been determined that the user has gazed at the toggle areas according to the predetermined pattern.

19. A method of changing content of a mobile device having a display with a plurality of toggle areas, the method comprising:

monitoring a gaze position of a user;

determining, based on the monitored gaze position, whether the user has gazed at the toggle areas according to a predetermined pattern;

activating an activation mode of the mobile device when it has been determined that the user has gazed at the toggle areas according to the predetermined pattern; and

changing the content of the mobile device based on the monitored gaze position of the user and a predetermined rule when the activation mode has been activated.

20. The method of claim 19, wherein the predetermined rule comprises scrolling content of the mobile device downward when the gaze position of the user is monitored as directed towards a bottom portion of the display of the mobile device.

21. The method of claim 19, wherein the predetermined rule comprises enlarging content displayed on a predetermined portion of the display of the mobile device when the gaze position of the user is monitored as directed towards the predetermined portion.