Calibrating real and virtual views

• US 7,369,101 B2
• Filed: 06/08/2004
• Issued: 05/06/2008
• Est. Priority Date: 06/12/2003
• Status: Active Grant

• Current Assignee: Siemens Medical Solutions USA Incorporated (Siemens AG)
• Original Assignee: Siemens Medical Solutions USA Incorporated (Siemens AG)
• Inventors: Sauer, Frank, Genc, Yakup, Navab, Nassir
• Primary Examiner(s): LEWIS, DAVID LEE
• Application Number:
  US10/863,414
• Publication Number:
  US 20060152434A1
• Time in Patent Office: 1,428 Days
• Field of Search: 345 4- 9, 345/419, 345/619, 345/629, 345/630, 359/629, 359/630, 359/632
• US Class Current: 345/8
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G02B 27/0189 Sight systems
• CPC Class Codes: G06F 3/012 Head tracking input arrangements

Abstract

A method for calibrating real and virtual views includes tracking a calibration screen, wherein a real reference point, generated by a real reference point generator, is projected on the calibration screen, aligning a view of a virtual reference point to a view of the real reference point in a display, wherein the real reference point generator and the display have a fixed relative position, determining a point correspondence between the virtual reference point and the real reference point, and determining one or more parameters for rendering a virtual object in the real scene.

Claims

1. An augmented reality system comprising:

a real reference generator for displaying a real reference on a calibration screen;

an optical see-through display having a fixed position with respect to the real reference generator;

a virtual reference generator for displaying a virtual reference on the optical see-through display;

an input device for aligning a view of the virtual reference with a view of the real reference through the optical see-through display, wherein the virtual reference is moved on the optical see-through display to be aligned with the view of the real reference; and

a processor for determining one or more parameters for rendering a virtual object as part of a real scene seen through the optical see-through display.

2. The augmented reality system of claim 1, further comprising a tracking camera for tracking a pose of the calibration screen with respect to the real reference.

3. The augmented reality system of claim 1, further comprising a tracking camera having a fixed position with respect to the real reference generator for capturing a view of the calibration screen.

4. The augmented reality system of claim 3, further comprising a processor, wherein an optical marker configuration is fixed to the calibration screen and imaged by the tracking camera, wherein the processor determines a positional relationship between the calibration screen and a head-mounted display according to a position of the optical marker configuration in an image captured by the tracking camera, the head-mounted display comprising the real reference generator and optical see-through display.

5. The augmented reality system of claim 1, further comprising at least one tracking camera for capturing a view of the calibration screen and a head-mounted display comprising the real reference generator and optical see-through display.

6. The augmented reality system of claim 5, further comprising a processor, wherein an optical marker configuration is fixed to each of the calibration screen and the head-mounted display and tracked by the at least one tracking camera, wherein the processor determines a positional relationship between the calibration screen and the head-mounted display according to the positions of respective optical marker configurations in the view captured by the at least one tracking camera.

7. A system for calibrating real and virtual views comprising:

a real reference generator for displaying a real reference on a calibration screen;

an optical display having a fixed position with respect to the real reference generator;

a virtual reference generator for generating a virtual reference in the optical display, wherein the virtual reference is independently movable with respect to the real reference;

an input device for aligning a view of the virtual reference with a view of the real reference, wherein the virtual reference is moved on the optical display with respect to the view of the real reference; and

a processor for determining one or more parameters for rendering a virtual object in a real scene seen in the optical display.

8. The system for calibrating real and virtual views claim 7, further comprising a camera capturing the view of the real reference, wherein the real reference is displayed in the optical display with the virtual reference superimposed thereon.

9. The system for calibrating real and virtual views claim 8, further comprising;

a tracking camera having a fixed position with respect to the real reference generator for capturing a view of the calibration screen; and

a processor, wherein an optical marker configuration is fixed to the calibration screen and tracked by the tracking camera, wherein the processor determines a positional relationship between the calibration screen and a head-mounted display according to the position of the optical marker configuration in the view captured by the tracking camera, the head-mounted display comprising the real reference generator and optical display.

10. The system for calibrating real and virtual views claim 7, further comprising a tracking camera coupled to the real reference generator for capturing a view of the calibration screen.

11. The system for calibrating real and virtual views claim 10, further comprising a processor, wherein an optical marker configuration is fixed to the calibration screen and tracked by the tracking camera, wherein the processor determines a positional relationship between the calibration screen and a head-mounted display according to the position of the optical marker configuration in the view captured by the tracking camera, the head-mounted display comprising the real reference generator and optical display.

12. The system for calibrating real and virtual views claim 7, further comprising a at least one tracking camera for capturing a view of the calibration screen and a head-mounted display comprising the real reference generator and optical display.

13. The system for calibrating real and virtual views claim 12, further comprising a processor, wherein an optical marker configuration is fixed to each of the calibration screen and the head-mounted display and tracked by the tracking camera, wherein the processor determines a positional relationship between the calibration screen and the head-mounted display according to the positions of respective optical marker configurations in the view captured by the at least one tracking camera.

14. A method for calibrating real and virtual views comprising:

tracking a calibration screen, wherein a real reference, generated by a real reference generator, is projected on the calibration screen;

aligning a virtual reference to a view of the real reference in a display, the virtual reference being independently movable with respect to the real reference, wherein the real reference generator and the display have a fixed relative position;

determining a point correspondence between the virtual reference and the real reference; and

determining one or more parameters for rendering a virtual object in the real scene.

15. The method for calibrating real and virtual views of claim 14, further comprising displaying the virtual reference on an optical see-through display, through which the real reference is visible.

16. The method for calibrating real and virtual views of claim 14, further comprising:

capturing a view of a real scene including the real reference; and

displaying the view of the real scene augmented with the virtual reference.