Virtual image registration in augmented display field

• US 20040080467A1
• Filed: 10/28/2002
• Published: 04/29/2004
• Est. Priority Date: 10/28/2002
• Status: Active Grant

• Current Assignee: University of Washington
• Original Assignee: University of Washington
• Inventors: Seibel, Eric, Furness, Thomas A. III, Chinthammit, Winyu
• Granted Patent: US 6,867,753 B2
• Time in Patent Office: Days
• US Class Current: 345/7
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

A virtual image is registered among a perceived real world background. Tracking light is scanned into the real world environment, which includes at least one detector pair. A first time and a second time at which the tracking light impinges on the first detector is detected, in which the first time and second time occurs within adjacent scan lines. A time at which a horizontal scan line edge (e.g., beginning of scan line or end of scan line) is encountered is derived as occurring one half way between the first time and the second time. The horizontal location of the first detector then is determined within a specific scan line inferring the scan line edge time. The vertical location of the detector is determined within a scan frame by measuring time duration using the beginning of the frame. By determining a location independently from the temporal resolution of the augmented imaging system, the temporal location of the detector is identified to a sub-pixel/sub-line precision. The augmented image is registered either to a 3D real world spatial coordinate system or to a time domain coordinate system based upon tracked position and orientation of the user.

Claims

1. A method of registering a virtual image among a perceived real world background, comprising:

scanning tracking light into the real world environment;

detecting a first time and a second time of impingement of the tracking light onto a first detector surface within the real world environment, said first time and second time occurring within adjacent scan lines;

determining a scan line edge time to be one half way between the first time and the second time;

defining the first detector location to a sub-pixel precision within a specific scan line based upon said first time and second time and

registering the virtual image among the real world background using the first detector location as a reference within said real world background.

2. The method of claim 1, further comprising:

detecting an amplitude of a first detector surface signal for multiple scan lines, and integrating the detected amplitude over the multiple scan lines to define the first detector location to a sub-line accuracy.

3. The method of claim 1, wherein said detecting, determining and defining are performed for a second detector surface and a third detector surface to define a second detector location and a third detector location, wherein said first detector surface, second detector surface and third detector surface are part of physically separate detectors;

said registering comprising registering the virtual image with six degrees of freedom among the real world background using the first detector location, second detector location and third detector location as references within said real world background.

4. The method of claim 1, wherein adjacent scan lines are first adjacent scan lines and said specific scan line is a first specific scan line, and further comprising:

detecting a first time and a second time of impingement of the tracking light onto two additional detectors within the real world environment, said second detector first time and second time occurring within second adjacent scan lines;

determining a scan line edge time between said second adjacent scan lines to be one half way between the second detector first time and the second time;

defining the second detector location to a sub-pixel precision within a second specific scan line based upon said first time and second time; and

registering the virtual image among the real world background with up to six degrees of freedom using the first detector location, second detector, and third detector location as references within said real world background.

5. The method of claim 1, in which said scanning is performed by a scanner, and further comprising:

tracking inertial movement of the scanner within the real world to define a spatial coordinate transformation between the scanner and the real world environment; and

calibrating the inertial tracking periodically based upon the defined first detector location.

6. The method of claim 1, further comprising:

scanning visible light along a plurality of scanning lines to generate the virtual image, wherein visible light output for a scan line is synchronized with a synchronization signal, said first detector location identified to sub-pixel accuracy by being identified independent of the synchronization signal.

7. The method of claim 6, wherein said visible light scanning and tracking light scanning are performed by a common scanner, said scanner subject to a phase drift of the scanning frequency, wherein said first detector location is identified to sub-pixel accuracy by identified being independent of the phase drift.

8. An augmented imaging system, comprising:

a scanner which scans tracking light into a real world environment;

means for detecting a first time and a second time of impingement of the tracking light onto a first detector within the real world environment, said first time and second time occurring within adjacent scan lines;

means for determining a scan line edge time to be one half way between the first time and the second time;

means for defining the first detector location to a sub-pixel precision within a specific scan line based upon said first time and second time; and

means for registering the virtual image among the real world background using the first detector location as a reference within said real world background.

9. The augmented imaging system of claim 8, in which the detecting means detects a first time and a second time of impingement of the tracking light onto a first detector;

said determining means determines a second detector scan line edge time to be one half way between the first time and the second time;

said defining means defines two additional detector locations to a sub-pixel precision within a specific scan line based upon said additional detectors first time and second time; and

said registering means registers the virtual image among the real world background with up to six degrees of freedom using the first detector location, second detector location, and third detector location as references within said real world environment.

10. The augmented imaging system of claim 8, further comprising:

means for tracking inertial movement of the scanner within the real world to define a spatial coordinate transformation between the scanner and the real world environment; and

means for calibrating the inertial tracking periodically based upon the defined first detector location.

11. The augmented imaging system of claim 8, in which the scanner scans visible light along a plurality of scanning lines to generate the virtual image, wherein visible light output for a scan line is synchronized with a synchronization signal, said first detector location identified to sub-pixel accuracy by being identified independent of the synchronization signal.

12. The augmented imaging system of claim 11, wherein said scanner is subject to a phase drift of the scanning frequency, and wherein said first detector location is identified to sub-pixel accuracy by being identified independent of the phase drift.

13. The augmented imaging system claim 8, further comprising:

means for detecting an amplitude of a first detector signal for multiple scan lines, and

means for integrating the detected amplitude over the multiple scan lines to define the first detector location to a sub-line accuracy.

14. An augmented imaging system, comprising:

an image source which generates image light modulated to create virtual image content;

a tracking light source which generates tracking light;

a scanner receiving the image light and the tracking light, the scanner deflecting the image light along a horizontal scan line, the image light being synchronized relative to a horizontal synchronization signal, the scanner concurrently deflecting the tracking light while deflecting the image light along the horizontal scan line, wherein a phase difference between the horizontal synchronization signal and the scanner frequency is subject to drift;

a plurality of detector pairs situated within the real world environment, each one of the plurality of detector pairs including a reference detector and another detector;

a detector which detects for each detector pair a first time and a second time of impingement of the tracking light onto each one reference detector, and detects impingement of the tracking light onto said another detector of each detector pair; and

a processor which identifies location of each one reference detector to a resolution which is less than a single pixel, said location determined independent of the horizontal synchronization signal and independent of said drift, wherein an edge of a horizontal scan line is inferred to occur at a time one-half way between said first time and second time for a given detector pair, said location of the reference detector of said given detector pair determined in time relative to the determined scan line edge.

15. The augmented imaging system of claim 14, wherein the processor registers the virtual image content relative to the location of at least one of the reference detectors.

16. The augmented imaging system of claim 14, further comprising:

an inertial tracker which tracks inertial movement of the scanner within the real world to define a spatial coordinate transformation between the scanner and the real world environment; and

wherein the processor calibrates the inertial tracking periodically based upon the defined first detector location.

17. The augmented imaging system of claim 16, wherein the processor registers the virtual image content relative to spatial coordinate system for the real world environment based upon said spatial coordinate transformation.

18. The augmented imaging system of claim 14, in which the detector detects an amplitude of a first detector signal for multiple scan lines, and wherein the processor integrates the detected amplitude over the multiple scan lines to define the first detector location to a sub-line accuracy.