Image manipulation based on tracked eye movement

• US 9,081,414 B2
• Filed: 06/23/2014
• Issued: 07/14/2015
• Est. Priority Date: 07/10/2009
• Status: Active Grant

• Current Assignee: Peking University
• Original Assignee: Peking University
• Inventors: Yuan, Xiaoru
• Primary Examiner(s): SIMPSON, LIXI CHOW
• Application Number:
  US14/311,443
• Publication Number:
  US 20140306883A1
• Time in Patent Office: 386 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Abstract

The disclosure relates to controlling and manipulating an image of an object on a display device based on tracked eye movements of an observer. When the object is displayed according to an initial view, the observer'"'"'s eye movement is tracked and processed in order to determine the focus of the observer'"'"'s attention or gaze on the image. Thereafter, the displayed image is modified to provide a better view of the part of the object in which the observer is most interested. This is accomplished by modifying at least one of the spatial positioning of the object within the viewing area, the angle of view of the object, and the viewing direction of the object.

Claims

1. A method, comprising:

obtaining a data point of a tracked eye gaze of a displayed three-dimensional image of an object in a viewing area, the data point representing a detected location of the tracked eye gaze within the viewing area, the data point obtained at a predetermined sampling interval;

mapping identified eye attentiveness points to a plurality of surface regions of the displayed three-dimensional image; and

rotating the displayed three-dimensional image based on the identified eye attentiveness points such that the region of the displayed three-dimensional image corresponding to a high degree of eye attentiveness is centered within the displayed three-dimensional image of the object.

2. The method of claim 1, further comprising determining a surface region of the plurality of surface regions with a highest degree of saliency based on the mapping.

3. The method of claim 2, wherein the rotating comprises rotating the object so that, in the rotated displayed three-dimensional image, a viewing direction of the object faces the determined surface region.

4. The method of claim 1, wherein the eye attentiveness points are identified by applying an unprojection process by multiplying an unprojection matrix with coordinates of a gaze location, the unprojection matrix is calculated based on a projection matrix used for projecting the surface of the object onto the viewing area.

5. A method, comprising:

obtaining a set of data points of a tracked eye gaze of a displayed image of an object in a viewing area, the set of data points representing a detected location of the tracked eye gaze within the viewing area, the set of data points obtained at a predetermined sampling interval;

analyzing the set of data points to identify a region of the displayed image corresponding to a high degree of eye attentiveness based on the tracked eye gaze; and

rotating the displayed image such that the region of the displayed image corresponding to the high degree of eye attentiveness is centered within the displayed image of the object.

6. The method of claim 5, further comprising determining a region of the viewing area in which the set of data points is located.

7. The method of claim 6, further comprising in response to determining that the region of the viewing area in which the set of data points is located is outside of the displayed image, rotating the displayed image toward the region of the viewing area.

8. The method of claim 5, further comprising determining whether a current viewing direction of the displayed image faces the region of the displayed image corresponding to the high degree of eye attentiveness.

9. The method of claim 5, wherein the analyzing comprises applying an unprojection process to the set of data points.

10. The method of claim 9, wherein the unprojection process comprises translating the set of data point to a three-dimensional space defining a surface of the object.

11. A method, comprising:

obtaining a set of data points of a tracked eye gaze of a displayed three-dimensional image of an object in a viewing area, the set of data points representing a detected location of the tracked eye gaze within the viewing area, the set of data points obtained at a predetermined sampling interval;

identifying eye attentiveness points on a surface of the displayed three-dimensional image by applying an unprojection process to translate the set of data points to a three dimensional space to define a surface of the object; and

rotating the displayed three-dimensional image based on the identified eye attentiveness points such that the region of the displayed three-dimensional image corresponding to a high degree of eye attentiveness is centered within the displayed three-dimensional image of the object.

12. The method of claim 11, further comprising determining a region of the viewing area in which the set of data points is located.

13. The method of claim 12, further comprising in response to determining that the region of the viewing area in which the set of data points is located is outside of the displayed image, rotating the displayed three-dimensional image toward the region of the viewing area.

14. The method of claim 11, further comprising determining whether a current viewing direction of the displayed image faces the region of the displayed image corresponding to the high degree of eye attentiveness.

15. A computing device having one or more computer processors that store executable instructions, operably connected to a display device, that in response to being executed cause the one or more computer processors to perform operations comprising:

obtaining a set of data points of a tracked eye gaze of a displayed image of an object in a viewing area, the set of data points representing a detected location of the tracked eye gaze within the viewing area, the set of data points obtained at a predetermined sampling interval;

analyzing the set of data points to identify a region of the displayed image corresponding to a high degree of eye attentiveness based on the tracked eye gaze; and

rotating the displayed image such that the region of the displayed image corresponding to the high degree of eye attentiveness is centered within the displayed image of the object.

16. The computing device of claim 15, further comprising determining a region of the viewing area in which the set of data points is located.

17. The computing device of claim 16, further comprising in response to determining that the region of the viewing area in which the set of data points is located is outside of the displayed image, rotating the displayed image toward the region of the viewing area.

18. The computing device of claim 15, further comprising determining whether a current viewing direction of the displayed image faces the region of the displayed image corresponding to the high degree of eye attentiveness.

19. The computing device of claim 15, wherein the analyzing comprises applying an unprojection process to the set of data points.

20. The computing device of claim 19, wherein the unprojection process comprises translating the set of data point to a three-dimensional space defining a surface of the object.