AUTOMATIC MAPPING OF AUGMENTED REALITY FIDUCIALS

• US 20100045701A1
• Filed: 08/24/2009
• Published: 02/25/2010
• Est. Priority Date: 08/22/2008
• Status: Abandoned Application

• Current Assignee: Cybernet Systems Corporation
• Original Assignee: Cybernet Systems Corporation
• Inventors: Scott, Katherine, Haanpaa, Douglas, Jacobus, Charles J.
• Application Number:
  US12/546,266
• Publication Number:
  US 20100045701A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Systems and methods expedite and improve the process of configuring an augmented reality environment. A method of pose determination according to the invention includes the step of placing at least one synthetic fiducial in a real environment to be augmented. A camera, which may include apparatus for obtaining directly measured camera location and orientation (DLMO) information, is used to acquire an image of the environment. The natural and synthetic fiducials are detected, and the pose of the camera is determined using a combination of the natural fiducials, the synthetic fiducial if visible in the image, and the DLMO information if determined to be reliable or necessary. The invention is not limited to architectural environments, and may be used with instrumented persons, animals, vehicles, and any other augmented or mixed reality applications.

Claims

1. A method of pose determination in an augmented reality system, comprising the steps of:

placing at least one synthetic fiducial in a real environment to be augmented;

providing a camera including apparatus for obtaining directly measured camera location and orientation (DLMO) information;

acquiring an image of the environment with the camera;

detecting the natural and synthetic fiducials;

estimating the pose of the camera using a combination of the natural fiducials, the synthetic fiducial if visible in the image, and the DLMO information if determined to be reliable or necessary.

2. The method of claim 1, wherein the synthetic fiducial is in the form of a bar code.

3. The method of claim 1, wherein the DLMO information is derived through inertial measurement.

4. The method of claim 1, wherein the DLMO information is derived through GPS or RF location measurement.

5. The method of claim 1, wherein the DLMO information is derived through sensor data.

6. The method of claim 1, wherein the DLMO information is obtained from an altimeter, accelerometer, gyroscope or magnetometer.

7. The method of claim 1, including the step of determining and recording the position of natural fiducials in the form of keyframes.

8. The method of claim 1, including the steps of:

determining the pose of a first fiducial A;

imaging the environment with a field of view including A and another fiducial, B; and

determining the pose of B using the pose of A and the offset between A and B.

9. The method of claim 8, including the steps of

imaging the environment with a field of view including B and another fiducial, C;

determining the pose of C using the pose of B and the offset between B and C; and

wherein the rotation and translation between A, B and C, if any, is represented by a quaternion.

11. The method of claim 1, including the steps of:

merging augmentations with the image of the real environment; and

presenting the augmented image to a viewer.

12. The method of claim 1, including the steps of:

generating augmentations in the form of synthetic 3D or 2D graphics, icons, or text;

merging augmentations with the image of the real environment; and

presenting the augmented image to a viewer.

13. The method of claim 1, including the steps of:

merging augmentations with the image of the real environment; and

presenting the augmented image to a viewer through a head-mounted display, touch screen display or portable computing or telecommunications device.

10. A method of determining the pose (position and orientation) of a plurality of fiducials in an environment, comprising the steps of:

determining the pose of a first fiducial, A;

imaging the environment with a field of view including A and a plurality of other fiducials; and

determining the pose of the other fiducials by batch optimizing the translation between some or all of the fiducials within the field of view, thereby eliminating the need for the DLMO information.

14. An augmented reality system, comprising:

a camera for imaging a real environment to be augmented, the camera including apparatus for obtaining directly measured camera location and orientation (DLMO) information;

at least one synthetic fiducial positioned in the environment; and

a processor operative to detect the natural and synthetic fiducials in the image acquired by the camera and estimate the pose of the camera using a combination of the natural fiducials, the synthetic fiducials if visible in the image, and the DLMO information if determined to be reliable or necessary.

15. The system of claim 14, wherein the synthetic fiducial is a bar code.

16. The system of claim 14, wherein the apparatus for obtaining the DLMO information is an inertial measurement system.

17. The system of claim 14, wherein the apparatus for obtaining the DLMO information is a GPS or RF location measurement system.

18. The system of claim 14, wherein the apparatus for obtaining the DLMO information is an altimeter, accelerometer, gyroscope or magnetometer or other sensor.

19. The system of claim 14, further including:

the same or a different processor for generating augmentations in the form of synthetic 3D or 2D graphics, icons, or text and merging the augmentations with the image of the real environment; and

a display for presenting the augmented image to a viewer.

20. The system of claim 19, wherein the display forms part of a head-mounted display, touch screen display or portable computing or telecommunications device.