System and method for measuring the registration accuracy of an augmented reality system

• US 7,190,331 B2
• Filed: 06/06/2002
• Issued: 03/13/2007
• Est. Priority Date: 06/06/2002
• Status: Active Grant

• Current Assignee: Siemens Corp. (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Navab, Nassir, Genc, Yakup, McGarrity, Erin, Tuceryan, Mihran
• Primary Examiner(s): Tung; Kee M.
• Assistant Examiner(s): Lay; Michelle K.
• Application Number:
  US10/166,226
• Publication Number:
  US 20030227470A1
• Time in Patent Office: 1,741 Days
• Field of Search: 345 7- 9, 345/173, 345/174, 345/633, 345/636, 345/638
• US Class Current: 345/9
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A method for determining registration accuracy of an augmented reality system is provided. The method includes the steps of providing an input device for inputting a position of a user'"'"'s touch; registering the input device with a workspace coordinate system of the augmented reality system; displaying a virtual object in a known position; recording a user'"'"'s touch position in response to the user attempting to touch the virtual object; and calculating a registration error of the augmented reality system by comparing the user'"'"'s touch position to the known position of the virtual object. The input device may be a passive device used in conjunction with a tracking system of the augmented reality system or an active device, such as a touch-sensitive device, coupled to the augmented reality system which outputs the user'"'"'s touch position.

Claims

1. In a system for augmenting a user'"'"'s view of real-world objects with virtual objects to provide a composite augmented reality image, the system including a display device for displaying the composite augmented reality image to the user, a tracking system for locating real-world objects, and a processor for determining the position and orientation of the user'"'"'s view based on the location of the real-world objects, for projecting the virtual objects onto the display device, and for processing a method for determining registration accuracy of the system comprising the steps of:

(a) calibrating the tracking system;

(b) calibrating the display device;

(c) providing an input device for inputting a position of the user'"'"'s touch;

(d) registering the input device with a coordinate system of the system;

(e) displaying a virtual object in a known position superimposed on said input device;

(f) recording a user'"'"'s touch position in response to the user attempting to touch the virtual object;

(g) calculating a registration error of the system by comparing the user'"'"'s touch position to the known position of the virtual object;

(h) repeating steps (e), (f), and (g), wherein for each repetition of the steps displaying the virtual object in a different known location; and

(i) reconstructing an error field of the system by determining residual distances between actual positions of the virtual objects and reconstructed positions of the virtual objects.

2. The method as in claim 1, further comprising the step of identifying errors of the system, wherein if the residual distances are small, the system error is structured and, if the residual distances are large, the system error is unstructured tracker noise.

3. The method as in claim 1, wherein the recording step further comprises the step of computing a three-dimensional position of the user'"'"'s touch position.

4. The method as in claim 1, further comprising the step of, after calculating a predetermined number of registration errors, recalibrating the display device and tracking system for an acceptable registration accuracy.

5. The method as in claim 1, further comprising the step of, after calculating a predetermined number of registration errors, modifying the current calibration of the display device and tracking system to account for the calculated errors.

6. The method as in claim 1, wherein the tracking system is a magnetic tracking system including a transmitter, a marker receiver and a digitizing stylus and the input device is a designated planar surface, the recording step further comprising determining the user'"'"'s touch position by determining a position of the stylus in relation to the planar surface by the tracking system.

7. The method as in claim 1, wherein the tracking system is a video-based infrared tracking system including a camera with an infrared filter and a ring of LEDs mounted to the camera and the input device is a graphic tablet including a set of fiducials with known geometry, the recording step further comprising determining the user'"'"'s touch position by determining a position of a user'"'"'s finger in relation to the graphic tablet by the tracking system.

8. The method as in claim 1, wherein the input device is a touch-sensitive device coupled to the processor.

9. In a system for augmenting a user'"'"'s view of real-world objects with virtual objects to provide a composite augmented reality image, the system including a display device for displaying the composite augmented reality image to the user, a tracking system for locating real-world objects, and a processor for determining the position and orientation of the user'"'"'s view based on the location of the real-world objects and for projecting the virtual objects onto the display device, an apparatus for determining registration accuracy of the system comprising:

an input device for determining a position of the user'"'"'s touch, the input device being registered with a coordinate system of the system, wherein when a virtual object is displayed in a known position superimposed on the input device, the user'"'"'s touch position is recorded in response to the user attempting to touch the virtual object, and the processor includes means for calculating a registration error of the system and means for comparing the user'"'"'s touch position to the known position of the virtual object.

10. The apparatus as in claim 9, wherein the processor computes a three-dimensional position of the user'"'"'s touch position.

11. The apparatus as in claim 9, wherein, after calculating a predetermined number of registration errors, the processor recalibrates the display device and tracking system for an acceptable registration accuracy.

12. The apparatus as in claim 9, wherein, after calculating a predetermined number of registration errors, the processor modifies the current calibration of the display device and tracking system to account for the calculated errors.

13. The apparatus as in claim 9, wherein the processor identifies structured errors of the system.

14. The apparatus as in claim 9, wherein the tracking system is a magnetic tracking system including a transmitter, a marker receiver and a digitizing stylus and the input device is a designated planar surface, wherein the user'"'"'s touch position is determined by a position of the stylus in relation to the planar surface by the tracking system.

15. The apparatus as in claim 9, wherein the tracking system is a video-based infrared tracking system including a camera with an infrared filter and a ring of LEDs mounted to the camera and the input device is a graphic tablet including a set of fiducials with known geometry, wherein the user'"'"'s touch position is determined by a position of a user'"'"'s finger in relation to the graphic tablet by the tracking system.

16. The apparatus as in claim 9, wherein the input device is a touch-sensitive device coupled to the processor.

17. The apparatus as in claim 9, wherein the input device is a touch-sensitive device coupled to a second processor, the second processor outputting a user'"'"'s touch position to the system processor.

18. The apparatus as in claim 9, wherein the user is requested to attempt to touch the virtual object.

19. In a system for augmenting a user'"'"'s view of real-world objects with virtual objects to provide a composite augmented reality image, the system including a display device for displaying the composite augmented reality image to the user, a tracking system for locating real-world objects, and a processor for determining the position and orientation of the user'"'"'s view based on the location of the real-world objects, for projecting the virtual objects onto the display device, and for processing a method for determining registration accuracy of the system comprising the steps of:

(a) providing an input device for inputting a position of the user'"'"'s touch;

(b) registering the input device with a coordinate system of the system;

(c) displaying a virtual object in a position known only to the processor superimposed on said input device;

(d) recording a user'"'"'s touch position in response to the user attempting to touch the virtual object; and

(e) calculating a registration error of the system by comparing the user'"'"'s touch position to the known position of the virtual object.