METHOD FOR CONTROLLING DEVICE ON THE BASIS OF EYEBALL MOTION, AND DEVICE THEREFOR

• US 20150091793A1
• Filed: 03/08/2013
• Published: 04/02/2015
• Est. Priority Date: 03/08/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Jun-seok, Yu, Kyung-hwa, Leigh, Sang-won, Chung, Jin-yong, Cho, Sung-goo
• Granted Patent: US 9,864,429 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris

Abstract

A method of controlling an operation of a display device using eye movements and a device for performing the method are provided. The method includes receiving eye movement information of a user; receiving blinking information of the user; generating a control command corresponding to the eye movement information and the blinking information; and controlling an operation of the display device based on the generated control command.

Claims

1. A method of controlling an operation of a display device based on eye movements of a user, the method comprising:

receiving eye movement information of a user;

receiving blinking information of the user;

generating a control command corresponding to the eye movement information and the blinking information; and

controlling an operation of the display device based on the generated control command,

wherein, in the generating of the control command, a position of a gaze of the user is checked based on the eye movement information and the control command is generated based on the position of the gaze.

2. The method of claim 1, further comprising dividing a display of the display device into a plurality of areas, and the generating comprises generating the control command based on an area gazed at by the user from among the divided areas and a movement direction of the gaze of the user from the area gazed at by the user.

3. The method of claim 2, wherein the dividing comprises dividing the display based on an area where contents are displayed on the display of the device.

4. The method of claim 1, wherein the generating comprises generating the control command based on a movement of a gaze of the user from the display to an area outside of the display.

5. The method of claim 1, further comprising:

displaying a control command selection window comprising a list of control commands on a display of the device;

sensing a selection of a control command from the control command list included in the control command selection window based on a user input via the displayed control command selection window; and

moving a cursor on the display of the device to a predetermined object based on the eye movement information of the user,

wherein the generating comprises generating a control command corresponding to the selected control command with respect to the object in response to eye blinking by the user.

6. The method of claim 1, further comprising:

fixing a position of a cursor corresponding to a gaze of the user; and

moving a position of the cursor to correspond to a moved position of the gaze of the user, in response to a movement of the gaze of the user.

7. The method of claim 1, further comprising:

displaying a previously set area around a cursor based on a user input; and

moving a cursor based on a position of a gaze of the user with respect to the displayed area.

8. The method of claim 1, further comprising:

moving a cursor to an object in response to the cursor being moved based on the eye movement information to an area that is located within a previously set range of the object on the display of the device.

17. A non-transitory computer readable medium comprising a computer program that is executable by a computer to perform the method of claim 1.

18. The method of claim 1, wherein the position of the gaze of the user and a movement direction of the gaze are determined based on the eye movement information, and

the control command is generated based on the position of the gaze, the movement direction of the gaze, and blinking information.

9. A device for controlling an operation of a device based on eye movements of a user, the device comprising:

an eye movement information receiver configured to receive information about eye movements of a user;

a blinking information receiver configured to receive information about eye blinking of the user; and

a control command generator configured to generate a control command corresponding to the eye movement information and the blinking information, and control an operation of the device based on the generated control command,

wherein the control command generating unit checks a position of a gaze of the user based on the eye movement information, and generates a control command based on the position of the gaze.

10. The device of claim 9, wherein the control command generator is configured to divide a display into a plurality of areas and generate a control command based on an area gazed at by the user from among the divided areas and a movement direction of the gaze of the user from the area gazed at by the user.

11. The device of claim 10, wherein the control command generator is configured to divide the display based on an area of the device where contents are displayed.

12. The device of claim 9, wherein the control command generator is configured to generate a control command based on a movement of a gaze of the user as the gaze of the user moves from a display to an area outside of the display.

13. The device of claim 9, further comprising a display configured to display a control command selection window including a list of control commands corresponding to eye blinking of the user,

and the device further comprises a sensor configured to sense a user input via the displayed control command selection window and eye blinking of the user to select a control command from the list, and the control command generator is configured to generate a control command corresponding to the sensed selected control command with respect to an object placed at a position of a cursor.

14. The device of claim 9, further comprising an error corrector configured to correct a position of a cursor on a display corresponding to a gaze of the user and match a position of a moved gaze of the user with the position of the cursor, in response to a movement of the gaze of the user.

15. The device of claim 9, further comprising:

a display configured to display a previously set area around a cursor based on a user input, and

the device further comprises an input adjuster configured to move the cursor with respect to the displayed area, based on a direction of a position at which the gaze of the user is located.

16. The device of claim 9, further comprising a display configured to move a cursor on the display to an object on the display, in response to the cursor being moved based on the eye movement information to an area that is located within a previously set range of the object on the display of the device.

19. The device of claim 9, wherein the control command generator is configured to determine the position of the gaze of the user and a movement direction of the gaze based on the eye movement information, and generate the control command based on the position of the gaze, the movement direction of the gaze, and blinking information.

20. An apparatus for controlling a display based on eye movement of a user, the apparatus comprising:

a receiver configured to receive eye movement information of a user, the eye movement information comprising blinking information and a position of a gaze of the user;

a controller configured to control data displayed on a display based on the received eye movement information.

21. The apparatus of claim 20, wherein the controller is configured to divide a display area of the display into a plurality of areas, and control the display based on a respective area from among the plurality of areas that corresponds to the position of the gaze of the user.

22. The apparatus of claim 20, wherein the controller is configured to control the display unit based on a speed and direction of a movement of the gaze of the user.

23. The apparatus of claim 20, wherein the controller is configured to control the display unit based on at least one of a length of time a user blinks or an amount of times the user blinks.

24. The apparatus of claim 20, wherein the controller is configured to move content displayed by the display in a direction that corresponds to a movement of the gaze of the user.