Image manipulation based on tracked eye movement

• US 8,797,261 B2
• Filed: 09/11/2013
• Issued: 08/05/2014
• Est. Priority Date: 07/10/2009
• Status: Active Grant

• Current Assignee: Peking University
• Original Assignee: Peking University
• Inventors: Yuan, Xiaoru
• Primary Examiner(s): SIMPSON, LIXI CHOW
• Application Number:
  US14/024,015
• Publication Number:
  US 20140028549A1
• Time in Patent Office: 328 Days
• Field of Search: 345/156
• US Class Current: 345/156
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Abstract

The disclosure relates to controlling and manipulating an image of an object on a display device based on tracked eye movements of an observer. When the object is displayed according to an initial view, the observer'"'"'s eye movement is tracked and processed in order to determine the focus of the observer'"'"'s attention or gaze on the image. Thereafter, the displayed image is modified to provide a better view of the part of the object in which the observer is most interested. This is accomplished by modifying at least one of the spatial positioning of the object within the viewing area, the angle of view of the object, and the viewing direction of the object.

Claims

1. A method, comprising:

obtaining a data point of a tracked eye gaze at a predetermined sampling interval of a displayed image of an object in a viewing area, the data point representing a detected location of the tracked eye gaze within the viewing area;

determining a range of the viewing area in which the data point is located;

magnifying the displayed image when the determined range is smaller than a first predetermined range; and

shrinking the displayed image when the determined range is larger than a second predetermined range.

2. The method of claim 1, further comprising analyzing the data point to identify a region of the displayed image corresponding to a high degree of eye attentiveness based on the tracked eye gaze.

3. The method of claim 2, further comprising modifying the displayed image by moving the object to thereby move the identified region of the object to a different location of the viewing area in the modified displayed image.

4. The method of claim 3, wherein the modifying includes moving the identified region of the object to the center of the viewing area.

5. The method of claim 1, wherein the first and second predetermined ranges include predetermined viewing angles.

6. The method of claim 5, further comprising computing a viewing angle of the viewing area based on the tracked eye gaze.

7. The method of claim 6, further comprising establishing the first and second predetermined ranges relative to the computed viewing angle of the entire viewing area.

8. The method of claim 1, wherein the first predetermined range is defined according to a first set of height and width parameters and the second predetermined range is defined according to a second set of height and width parameters.

9. A method, comprising:

obtaining a data point of a tracked eye gaze at a predetermined sampling interval of a displayed three-dimensional image of an object in a viewing area, the data point representing a detected location of the tracked eye gaze within the viewing area;

identifying eye attentiveness points on a surface of the displayed three-dimensional image by applying an unprojection process to translate the obtained data point to a point on the displayed three-dimensional image; and

modifying the displayed three-dimensional image based on the identified eye attentiveness points.

10. The method of claim 9, further comprising mapping the identified eye attentiveness points to a plurality of surface regions of the displayed three-dimensional image.

11. The method of claim 10, further comprising determining a surface region of the plurality of surface regions with a highest degree of saliency based on the mapping.

12. The method of claim 10, further comprising modifying the displayed three-dimensional image by rotating the object so that, in the modified displayed three-dimensional image, the viewing direction of the object faces the determined surface region.

13. The method of claim 9, wherein the unprojection process is applied by multiplying an unprojection matrix with coordinates of a gaze location, the unprojection matrix is calculated based on a projection matrix used for projecting the surface of the object onto the viewing area.

14. A computing device having one or more computer processors that store executable instructions, operably connected to a display device, that when executed cause the one or more computer processors to perform operations comprising:

obtaining a data point of a tracked eye gaze at a predetermined sampling interval of a displayed three-dimensional image of an object in a viewing area, the data point representing a detected location of the tracked eye gaze within the viewing area;

identifying eye attentiveness points on a surface of the displayed three-dimensional image by applying an unprojection process to translate the obtained data point to a point on the displayed three-dimensional image; and

modifying the displayed three-dimensional image based on the identified eye attentiveness points.

15. The computing device of claim 14, wherein the instructions cause the one or more computer processors to perform further operations including mapping the identified eye attentiveness points to a plurality of surface regions of the displayed three-dimensional image.

16. The computing device of claim 14, wherein the operations further comprise determining a surface region of the plurality of surface regions with a highest degree of saliency based on the mapping.

17. The computing device of claim 14, wherein the operations further comprise modifying the displayed three-dimensional image by rotating the object to face, the viewing direction of the displayed object to the determined surface region in the modified displayed three-dimensional image.

18. The computing device of claim 14, wherein the unprojection process is applied by multiplying an unprojection matrix with coordinates of a gaze location, the unprojection matrix is calculated based on a projection matrix used for projecting the surface of the object onto the viewing area.