Method and apparatus for augmented reality hybrid tracking system with fiducial-based heading correction

• US 20040080548A1
• Filed: 10/29/2002
• Published: 04/29/2004
• Est. Priority Date: 10/29/2002
• Status: Active Grant

• Current Assignee: Xlabs Holdings LLC
• Original Assignee: Xlabs Holdings LLC
• Inventors: Daily, Michael J., Azuma, Ronald T., Neely, Howard E. III, Isdale, Gerald B.
• Granted Patent: US 7,120,875 B2
• Time in Patent Office: Days
• US Class Current: 345/850
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

An apparatus and method is disclosed for enabling accurate registration of virtual information upon real objects in selected environments from the perspective of a moveable observation platform having a position and an orientation within an observation space positioned within the environment, the position and orientation defining the registration of the virtual information within the observation space within the environment, which may comprise: a plurality of at least three tracking fiducials selectively each respectively located in fixed predetermined locations in the observation space; the observation platform including a position sensor apparatus adapted to sense an estimated position of the observation platform in the observation space and a heading sensor apparatus adapted to sense an estimated heading of the observation platform within the observation space; the observation platform further including a wide view fiducial detection apparatus mounted on the observation platform and adapted to detect the relative direction from the observation platform to each of the plurality of tracking fiducials, relative to a heading direction of the observation platform, assuming an approximated heading direction of the observation platform, the wide view fiducial detection apparatus adapted further to detect fiducials in a panoramic field of view greater than at least one hundred and eight degrees; a heading error detection apparatus adapted to detect the difference between the estimated heading and the present heading based upon the observed relative directions to the respective tracking fiducials, and forming a corrected heading from the estimated heading and the detected heading error; a position error detection apparatus adapted to detect the difference between the estimated position in the observation space and the actual position based upon the observed positions of the respective fiducials and the corrected heading of the observation platform. The apparatus and method may further comprise the estimated heading is derived from at least the output of a rotation sensor and from a prior corrected position determination if a prior corrected position determination is available; the estimated heading is a prior corrected heading determination; the estimated position is derived from at least the output of a position detector and from a prior corrected position determination if a prior corrected position determination is available or the estimated position is a prior corrected position determination.

Claims

1. An apparatus for enabling accurate registration of virtual information upon real objects in selected environments from the perspective of a moveable observation platform having a position and an orientation within an observation space positioned within the environment, the position and orientation defining the registration of the virtual information within the observation space within the environment, comprising:

a plurality of at least three tracking fiducials selectively each respectively located in fixed predetermined locations in the observation space;

the observation platform including a position sensor apparatus adapted to sense an estimated position of the observation platform in the observation space and a heading sensor apparatus adapted to sense an estimated heading of the observation platform within the observation space;

the observation platform further including a wide view fiducial detection apparatus mounted on the observation platform and adapted to detect the relative direction from the observation platform to each of the plurality of tracking fiducials, relative to a heading direction of the observation platform, assuming an approximated heading direction of the observation platform, the wide view fiducial detection apparatus adapted further to detect fiducials in a panoramic field of view greater than at least one hundred and eighty degrees;

a heading error detection apparatus adapted to detect the difference between the estimated heading and the present heading based upon the observed relative directions to the respective tracking fiducials, and forming a corrected heading from the estimated heading and the detected heading error;

a position error detection apparatus adapted to detect the difference between the estimated position in the observation space and the actual position based upon the observed positions of the respective fiducials and the corrected heading of the observation platform.

2. The apparatus of claim 1 wherein the estimated heading is derived from at least the output of a rotation sensor and from a prior corrected position determination if a prior corrected position determination is available.

3. The apparatus of claim 1 wherein the estimated heading is a prior corrected heading determination.

4. The apparatus of claim 2 wherein the estimated position is derived from at least the output of a position detector and from a prior corrected position determination if a prior corrected position determination is available.

5. The apparatus of claim 3 wherein the estimated position is a prior corrected position determination.

6. The apparatus of claim 4 wherein the estimated position is a prior corrected position determination.

7. The apparatus of claim 1 wherein the panoramic fiducial detection apparatus includes a camera having a greater field of vision that the field of vision of a human user of the camera.

8. The apparatus of claim 1 wherein the panoramic fiducial detection apparatus includes a camera having a three hundred and sixty degree field of vision coverage.

9. An apparatus for enabling accurate registration of virtual information upon real objects in selected environments from the perspective of a moveable observation platform having a position and an orientation within an observation space positioned within the environment, the position and orientation defining the registration of the virtual information within the observation space within the environment, comprising:

a plurality of at least three tracking fiducials selectively each respectively located in fixed predetermined locations in the observation space;

the observation platform including a position sensing means for sensing an estimated position of the observation platform in the observation space and a heading sensing means for sensing an estimated heading of the observation platform within the observation space;

the observation platform further including a wide view fiducial detection means mounted on the observation platform for detecting the relative direction from the observation platform to each of the plurality of tracking fiducials, relative to a heading direction of the observation platform, assuming an approximated heading direction of the observation platform, the wide view fiducial detection means, including means for detecting fiducials in a panoramic field of view greater than at least one hundred and eighty degrees;

a heading error detection means for detecting the difference between the estimated heading and the present heading based upon the observed relative directions to the respective tracking fiducials, and forming a corrected heading from the estimated heading and the detected heading error;

a position error detection means for detecting the difference between the estimated position in the observation space and the actual position based upon the observed positions of the respective fiducials and the corrected heading of the observation platform.

10. The apparatus of claim 9 wherein the estimated heading is derived from at least the output of a rotation sensor and from a prior corrected position determination if a prior corrected position determination is available.

11. The apparatus of claim 9 wherein the estimated heading is a prior corrected heading determination.

12. The apparatus of claim 10 wherein the estimated position is derived from at least the output of a position detector and from a prior corrected position determination if a prior corrected position determination is available.

13. The apparatus of claim 11 wherein the estimated position is a prior corrected position determination.

14. The apparatus of claim 12 wherein the estimated position is a prior corrected position determination.

15. The apparatus of claim 9 wherein the panoramic fiducial detection means includes a camera having a greater field of vision that the field of vision of a human user of the camera.

16. The apparatus of claim 9 wherein the panoramic fiducial detection means includes a camera having a three hundred and sixty degree field of vision coverage.

17. A method for enabling accurate registration of virtual information upon real objects in selected environments from the perspective of a moveable observation platform having a position and an orientation within an observation space positioned within the environment, the position and orientation defining the registration of the virtual information within the observation space within the environment, comprising:

providing a plurality of at least three tracking fiducials selectively each respectively located in fixed predetermined locations in the observation space;

the observation platform including a position sensing means for sensing an estimated position of the observation platform in the observation space and a heading sensing means for sensing an estimated heading of the observation platform within the observation space;

the observation platform further including a wide view fiducial detection means mounted on the observation platform for detecting the relative direction from the observation platform to each of the plurality of tracking fiducials, relative to a heading direction of the observation platform, assuming an approximated heading direction of the observation platform, the wide view fiducial detection means, including means for detecting fiducials in a panoramic field of view greater than at least one hundred and eighty degrees;

detecting the difference between the estimated heading and the present heading based upon the observed relative directions to the respective tracking fiducials, and forming a corrected heading from the estimated heading and the detected heading error;

detecting the difference between the estimated position in the observation space and the actual position based upon the observed positions of the respective fiducials and the corrected heading of the observation platform.

18. The method of claim 17 including the step of deriving the estimated heading from at least the output of a rotation sensor and from a prior corrected position determination if a prior corrected position determination is available.

19. The method of claim 17 including the step of utilizing as the the estimated heading a prior corrected heading determination.

20. The method of claim 18 including, the step of deriving the estimated position from at least the output of a position detector and from a prior corrected position determination if a prior corrected position determination is available.

21. The apparatus of claim 19 including the step of utilizing as the estimated position a prior corrected position determination.

22. The apparatus of claim 20 including the step of utilizing as the estimated position a prior corrected position determination.

23. The apparatus of claim 17 including the step of utilizing in the panoramic fiducial detection means a camera having a greater field of vision that the field of vision of a human user of the camera.

24. The apparatus of claim 17 wherein including the step of utilizing in the panoramic fiducial detection means a camera having a three hundred and sixty degree field of vision coverage.