Process for anticipation and tracking of eye movement

• US 6,574,352 B1
• Filed: 05/18/1999
• Issued: 06/03/2003
• Est. Priority Date: 05/18/1999
• Status: Expired due to Term

• Current Assignee: Rockwell Collins Simulation and Training Solutions, LLC (Rtx Corporation)
• Original Assignee: Evans & Sutherland Computer Corp. (Elevate Entertainment, Inc.)
• Inventors: Skolmoski, Philip
• Primary Examiner(s): Johnson, Timothy M.
• Assistant Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US09/314,425
• Time in Patent Office: 1,477 Days
• Field of Search: 382/103, 382/108, 382/131, 382/154, 382/173, 382/204, 345/586, 345/634, 345/848, 345/441, 345/582, 345/588, 348/115, 348/42, 348/116, 348/121, 348/122, 348/123, 348/129, 348/128, 348/169, 348/120, 348/172, 340/988, 340/990, 434/44, 434/30
• US Class Current: 382/103
• CPC Class Codes: G09B 9/307 by helmet-mounted projector or display G09B9/36 takes precedence; optical parts of head-up displays G02B27/01

Abstract

A method for anticipation and tracking eye movement for head tracked projectors which divides a projection surface into spherical triangles and uses interpolating calibration values stored at each vertex of the triangle. A calibration map is used that contains information about the relationship between the user'"'"'s head orientation and where they are looking with their eyes at each calibration point. The projection surface is divided into spherical triangles which are searched to find the triangle which contain the user'"'"'s interpolated view point by performing a dot product test between the interpolated view point and the unit normal vectors inside the three planes that make up the sides of the spherical triangle extended back to the eye point. When a dot product test fails for any side of the triangle the pointer is followed to the adjacent triangle where the test starts over. When the triangle is found that has the interpolated view point inside it, the selected triangle is divided into three sub-triangles. Then the area of the sub-triangles inside the selected triangle is computed. The areas of the sub-triangles are then used to find ratios of the sub-triangles'"'"' areas versus the selected spherical triangle area for weighting the calibration points stored with each vertex. Then the weighted calibrations are added to the values for the interpolated view point. This creates a calibrated view point where the head tracked projector is then pointed and the computer generated simulated image is projected.

Claims

1. A method for anticipation and tracking of a user'"'"'s eye movements in a head tracked image generation system, comprising the steps of:

(a) defining a viewing surface segmented into a plurality of contiguous triangles across the viewing surface upon which a head tracked image generator displays computer generated images, wherein the contiguous triangles are selected from the group consisting of planar triangles, spherical triangles and contoured triangles;

(b) identifying a plurality of calibration values for vertices of the triangles;

(c) developing a value from the calibration values representing a difference between (i) a user'"'"'s head orientation and (ii) where a user is actually looking;

(d) interpolating a user'"'"'s view point onto the viewing surface to determine an interpolated view point where the images should be displayed on the viewing surface;

(e) searching the plurality of triangles to find a selected triangle which contains the interpolated view point;

(f) dividing the selected triangle containing the view point into three contiguous sub-triangles with common edges extending from three vertices of the selected triangle to a common vertex point corresponding to the view point;

(g) comparing surface areas of the sub-triangles with the selected triangle; and

(h) correlating comparative surface areas with the calibration values to identify a proper position for the computer generated images.

2. The method of claim 1 wherein the step of comparing surface areas of the sub-triangles with the selected triangle further comprises calculating surface area ratios of the three sub-triangles in relation to the surface area of the selected triangle.

3. The method of claim 2 wherein the step of correlating comparative surface areas with the calibration values further comprises the steps of:

multiplying the surface area ratios of the sub-triangles with the calibration values at each vertex opposite the sub-triangles in the selected triangle to create weighted calibration values; and

adding the weighted calibration values to the interpolated view point to allow the head tracked projector to use a calibrated interpolation point to ascertain where the computer generated images should be displayed on the projection surface.

4. The method of claim 3 wherein the step of multiplying the surface area ratios of the sub-triangles with the calibration values at each vertex of the original triangle to create weighted calibration values further comprises the step of executing the equation:

5. The method of claim 1 wherein the step of searching the plurality of triangles to find a selected triangle which contains the interpolated view point, further comprises the steps of:

selecting a current triangle as a starting point for a search;

testing to see if the interpolated view point is within the current triangle;

moving to a next adjacent triangle when the interpolated view point is not within the current triangle; and

selecting the current triangle when the interpolated view point is within the current triangle.

6. The method of claim 5 wherein the step of testing to see if the interpolated viewing point is within the current triangle further comprises the steps of:

forming three planes which extend between a user'"'"'s nominal eye point and each pair of vertices in the current triangle, wherein each plane has a unit normal vector and the planes form a pyramid shaped closed volume having the nominal eye point as the apex;

performing a dot product test between the interpolated view point and the unit normal of each plane to create a signed scalar value; and

comparing a sign of the scalar value with a sign of the unit normal vector such that when the interpolated view point is within the triangle, the sign associated with each unit normal vector will match the sign of the interpolated point.

7. The method of claim 5 wherein the step of selecting a current triangle as a starting point for a search further comprises selecting a triangle as a starting point for a search which contained the interpolated view point in a previous computer generated image frame.

8. The method of claim 1 wherein the step of comparing surface areas of the sub-triangles with the selected triangle further comprises of the step of finding the surface areas of the selected triangle and each of its sub-triangles.

9. The method of claim 8 wherein the step of finding the surface areas of the selected triangle and each of its sub-triangles further comprises the steps of:

calculating an angle between each pair of vertices in the selected triangle;

forming three cords between each of the selected triangles vertices and the interpolated view point;

finding the approximate length of a cord between each pair of vertices;

finding the angles for each of the three vertices;

computing the area of the selected triangle; and

computing the areas of the sub-triangles.

10. The method of claim 8 wherein the step of finding the surface areas of the selected triangle and each of its sub-triangles further comprises the step of executing the equation:

11. The method of claim 1 wherein the step of comparing surface areas of the sub-triangles with the selected triangle further comprises finding the surface area ratios of the sub-triangles by taking the area of each of the three sub-triangles and dividing by the area of the selected triangle.

12. The method of claim 1 wherein the triangles provided with the projection surface are less the ⅛

of the projection surface size.

13. The method of claim 1 wherein step of providing a plurality of triangles further comprises providing a plurality of triangles wherein each triangle is adjacent to at least one other triangle.

14. The method of claim 1 wherein the step of providing a plurality of triangles further comprises the step of providing a plurality of triangles which are bounded by a plane and each triangle is bounded by at least one other triangle and 2 planes.

15. The method of claim 1 wherein the step of providing a plurality of triangles further comprises the step of providing a plurality of triangles which are bounded by a plane and each triangle is bounded by at least two other triangles and 1 plane.

16. The method of claim 1 wherein the triangle vertices are represented in the Cartesian coordinate system.

17. The method of claim 1 wherein the representation of the triangles in the Cartesian coordinate system have an origin set at the nominal eye point.

18. The method of claim 1 wherein the vertices of the triangles are represented in the azimuth and elevation coordinate system.