Method and apparatus for multi-level image alignment

• US 6,433,840 B1
• Filed: 07/22/1999
• Issued: 08/13/2002
• Est. Priority Date: 07/22/1999
• Status: Expired due to Term

• Current Assignee: Rockwell Collins Simulation and Training Solutions, LLC
• Original Assignee: Evans & Sutherland Computer Corp. (Elevate Entertainment, Inc.)
• Inventors: Poppleton, Kenneth A.
• Primary Examiner(s): Brier, Jeffery
• Assistant Examiner(s): YANG, RYAN R
• Application Number:
  US09/358,939
• Time in Patent Office: 1,118 Days
• Field of Search: 348/745, 348/746, 348/747, 348/190, 348/806, 348/807, 345/647, 345/611, 315/368.13, 315/368.11, 315/368.16
• US Class Current: 348/745
• CPC Class Codes: H04N 3/2335 with calculating means

Abstract

A method for anticipation and tracking eye movement for head tracked projectors which divides a projection surface into spherical triangles and uses interpolating calibration values stored at each vertex of the triangle. A calibration map is used that contains information about the relationship between the user'"'"'s head orientation and where they are looking with their eyes at each calibration point. The projection surface is divided into spherical triangles which are searched to find the triangle which contain the user'"'"'s interpolated view point by performing a dot product test between the interpolated view point and the unit normal vectors inside the three planes that make up the sides of the spherical triangle extended back to the eye point. When a dot product test fails for any side of the triangle the pointer is followed to the adjacent triangle where the test starts over. When the triangle is found that has the interpolated view point inside it, the selected triangle is divided into three sub-triangles. Then the area of the sub-triangles inside the selected triangle is computed. The areas of the sub-triangles are then used to find ratios of the sub-triangles'"'"' areas versus the selected spherical triangle area for weighting the calibration points stored with each vertex. Then the weighted calibrations are added to the values for the interpolated view point. This creates a calibrated view point where the head tracked projector is then pointed and the computer generated simulated image is projected.

Claims

1. A method for multi-level image alignment using a base grid with control points, comprising the steps of:

(a) creating a delta surface having a plurality of control points equal to a number of control points in the base grid wherein the delta surface control point values are set to zero;

(b) defining a control grid with a desired number of control points from the base grid;

(c) entering an adjustment value for a selected control point in the control grid;

(d) calculating derivatives at each point in the control grid in both U and V directions;

(e) interpolating the delta surface based on each control point'"'"'s derivatives to generate a modified delta surface;

(f) adding the delta surface to the base surface to form an adjusted raster surface for display.

2. A method for multi-level image alignment in a raster image display system, comprising the steps of:

(a) generating a base grid with a plurality of adjustment points which a user can interactively adjust;

(b) providing multiple grid levels wherein each grid level contains a sub-set of the plurality of adjustment points from the base grid;

(c) selecting a grid level and the adjustment points to modify;

(d) adjusting selected adjustment points to allow image distortion errors to be corrected;

(e) choosing progressively finer grid levels of adjustment wherein each grid level has adjustment points and the points at finer grid levels are automatically adjusted to align in relation to points adjusted at coarser grid levels;

(f) adjusting the progressively finer grid levels until a desirable adjustment has been reached; and

(g) incorporating grid adjustments into the base raster surface.

3. The method as in claim 2 wherein step (b) of providing multiple grid levels further comprises the steps of:

creating a 2×

2 grid having adjustment points presented to the user for interactive user adjustment; and

adjusting any of the points present on the grid which automatically adjusts finer adjustment points in the base grid which cannot be adjusted in the 2×

2 grid.

4. The method of claim 3 wherein the method further comprises the steps of:

generating a 3×

3 adjustment grid to present to the user for interactive user adjustment; and

adjusting any of the points in the 3×

3 grid which automatically adjusts the finer adjustment points in the base grid.

5. The method as in claim 2 wherein the step of choosing progressively finer grid levels further comprises the step of choosing progressively finer grid levels by dividing the previous subdivisions on the grid level by half.

6. The method as in claim 2 wherein the step of choosing progressively finer grid levels further comprises the step of choosing progressively finer grid levels by dividing previous grid level subdivisions into smaller sub-divisions.

7. The method as in claim 2 wherein step (b) of providing multiple grid levels further comprises the steps of:

creating a first grid level which is a 2×

2 grid; and

providing subsequent grid levels with adjustment points by dividing grid areas by a selected numerical factor.

8. The method as in claim 2 wherein step (b) of providing multiple levels of grid control points further comprises the steps of:

providing a first level grid which is a 2×

2 grid;

providing a second level grid which is a 3×

3 grid;

providing a third level grid which is a 5×

5 grid; and

providing a fourth level grid which is a 9×

9 grid.

9. A method for multi-level image alignment, the method comprising the steps of:

(a) generating a 2×

2 grid having adjustment points to present to the user for interactive user adjustment;

(b) adjusting any of the points present on the grid which in turn automatically adjusts finer adjustment points through the 2×

2 grid;

(c) generating a 3×

3 adjustment grid to provide to the user for interactive user adjustment;

(d) adjusting any of the points on the grid which automatically adjusts finer adjustment points which cannot be adjusted at the 3×

3 adjustment level;

(e) creating progressively finer grid levels of adjustment each having adjustment points wherein the finer points are adjusted relative to the points adjusted at coarser grid levels;

(f) adjusting the progressively finer grid levels until the finest grid level has been reached and adjusted; and

(g) incorporating the grid adjustments into a base raster surface.

10. An apparatus for multi-level image alignment in a raster image display system with a user interface, comprising:

a coarse grid with a plurality of adjustment points provided to a user through the user interface wherein a user can interactively adjust the points to correct distortion in a raster based image; and

a plurality of control grids having control points provided to the user through the user interface wherein each control grid has progressively more points than the coarse grid and the previous control grid, so the user can adjust the points which also adjusts the raster image projected by the raster image display system.

11. The apparatus of claim 10 further comprising a selection means for selecting one of the control grids provided through the user interface wherein a desired level of adjustment with an appropriate number of points in the grid can be adjusted.

12. The apparatus of claim 10 further comprising a cathode ray tube (CRT) projection device for which the raster image is to be adjusted and projected through for viewing.

13. The apparatus of claim 10 wherein the coarse grid is selected from the group consisting of a 2×

2 grid, 3×

3 grid, 5×

5 grid, 7×

7 grid and a 9×

9 grid, wherein each of the points presented in the coarse grid automatically adjusts points at a finer grid level.

14. The apparatus as in claim 13 wherein the coarse grid can be further broken down into progressively finer control grid levels by dividing control grid areas by half to provide additional control grid points which can be individually adjusted.

15. The apparatus as in claim 13 further comprising a selection means for selecting a desired control point for adjustment.

16. The apparatus as in claim 10 wherein each control grid and coarse grid is represented as a bicubic spline surface.

17. The apparatus as in claim 10 further comprising a delta surface with a number of control points equal to the control points in a currently selected control grid wherein each of point in the delta surface has not been initially adjusted and then adjustment values are added to the delta surface points to create adjusted control points.

18. The apparatus as in claim 17 further comprising an interpolation means to interpolate between unadjusted points in the delta surface and adjusted control points by calculating derivatives at each control point with respect to an adjusted control point and then adjusting the delta surface points to create a smooth curve between the adjusted point and previously unadjusted points, and then adding the delta surface to a base surface to form an adjusted raster surface which is used by the raster image display system.

19. The apparatus as in claim 10 further comprising a base surface to store all of the adjustments made to the control points.

20. An apparatus for multi-level image alignment in a raster image display system with a user interface, comprising:

a coarse grid with a plurality of adjustment points provided to a user through the user interface wherein a user can interactively adjust the points to correct distortion in a raster based image; and

a plurality of control grids having control points provided to the user through the user interface wherein each control grid has progressively more points than the coarse grid and the previous control grid, so the user can adjust the points which also adjusts the raster image projected by the raster image display system; and

an intermediate surface to temporarily store adjusted control points and interpolated surface curvatures, wherein values in the intermediate storage surface are added to a base surface and then the intermediate surface is reset to zero.