Accuracy evaluation of video-based augmented reality enhanced surgical navigation systems

• US 20050215879A1
• Filed: 03/14/2005
• Published: 09/29/2005
• Est. Priority Date: 03/12/2004
• Status: Abandoned Application

• Current Assignee: Bracco Imaging S.p.A. (Bracco SPA)
• Original Assignee: Bracco Imaging S.p.A. (Bracco SPA)
• Inventors: Chuanggui, Zhu
• Application Number:
  US11/080,172
• Publication Number:
  US 20050215879A1
• Time in Patent Office: Days
• US Class Current: 600/407
• CPC Class Codes: A61B 2017/00725 Calibration or performance testing
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

Systems and methods for measuring overlay error in a video-based augmented reality enhanced surgical navigation system are presented. In exemplary embodiments of the present invention the system and method include providing a test object, creating a virtual object which is a computer model of the test object, registering the test object, capturing images of control points on the test object at various positions within an augmented reality system'"'"'s measurement space, and extracting positions of control points on the test object from the captured images, calculating the positions of the control points in virtual image, and calculating the positional difference of positions of corresponding control points between the respective video and virtual images of the test object. The method and system can further assess if the overlay accuracy meets an acceptable standard. In exemplary embodiments of the present invention a method and system are provided to identify the various sources of error in such systems and assess their effects on system accuracy. In exemplary embodiments of the present invention, after the accuracy of an AR system is determined, the AR system may be used as a tool to evaluate the accuracy of other processes in a given application, such as registration error.

Claims

1. A method of measuring overlay error in augmented reality systems, comprising:

providing a test object;

registering the test object;

capturing images of one or more reference points on the test object at various positions within a defined workspace;

extracting positions of reference points on the test object from the captured images;

calculating re-projected positions of the reference points; and

calculating the differences between the extracted and re-projected reference points.

2. The method of claim 1, wherein the test object is one of planar, bi-planar, volumetric or comprising a single point.

3. The method of claim 1, wherein the test object is moved within the defined workspace by precisely known increments to acquire multiple positions for each of the reference points.

4. The method of claim 1, wherein the test object is precisely manufactured or measured such that the distances between successive reference points are substantially equal to within known tolerances.

5. The method of claim 1, wherein the test object has one or more pivots, and wherein the distances from said pivots to the reference points are precisely known to within defined tolerances.

6. The method of claim 3, wherein at least three positions for each reference point are acquired.

7. The method of claim 1, wherein calculation of the differences between the extracted and re-projected reference points is as to each reference point and includes calculation of one or more of a minimum, maximum, mean and standard deviation over all reference points within the defined workspace.

8. The method of claim 1, further comprising determining whether given the overall differences between all of the extracted and re-projected reference points the augmented reality system meets a given standard.

9. The method of claim 1, further comprising using the overall differences between all of the extracted and re-projected reference points as a baseline against which to measure other sources of overlay error.

10. The method of claim 9, wherein said other sources of overlay error include registration error.

12. The method of claim 1, wherein the test object is one of planar, bi-planar, volumetric or comprising a single point.

25. The method of claim 1, wherein the defined workspace is a space associated with the camera or imaging system.

11. A method of measuring overlay error in augmented reality systems, comprising:

providing a real test object;

generating a virtual test object;

registering the real test object to the virtual test object;

capturing images of one or more reference points on the test object and generating virtual images of corresponding points on the virtual test

object at various positions within a defined workspace;

extracting positions of reference points on the real test object from the captured images;

extracting corresponding positions of said reference points on the virtual test object from the virtual images; and

calculating the positional differences between the real and virtual reference points.

13. The method of claim 11, wherein the test object is moved within the defined workspace by precisely known increments to acquire multiple positions for each of the reference points.

14. The method of claim 11, wherein the test object is precisely manufactured or measured such that the distances between successive reference points are substantially equal to within known tolerances.

15. The method of claim 11, wherein the test object has one or more pivots, and wherein the distances from said pivots to the reference points are precisely known to within defined tolerances.

16. The method of claim 13, wherein at least three positions for each reference point are acquired.

17. The method of claim 11, wherein calculation of the differences between the extracted and re-projected reference points is as to each reference point and includes calculation of one or more of a minimum, maximum, mean and standard deviation over all reference points within the defined workspace.

18. A system for measuring overlay error in an augmented reality system, comprising:

a test object with one or more defined reference points;

a tracking device;

a data processor;

a camera or imaging device used in the AR system,

wherein the test object and camera can each be tracked in a tracking space of the tracking system, and wherein in operation the camera or imaging system generates one or more images of the test object and the data processor generates an equal number of virtual images of a corresponding virtual test object at various positions in a defined workspace and locational differences between corresponding reference points are calculated.

19. The system of claim 18, wherein the test object is one of panar, bi-planar, volumetric or comprising a single point.

20. The system of claim 18, wherein in operation the test object is moved within the defined workspace by precisely known increments to acquire multiple positions for each of the reference points.

21. The system of claim 18, wherein the test object is precisely manufactured or measured such that the distances between successive reference points are substantially equal to within known tolerances.

22. The system of claim 18, wherein the test object has one or more pivots, and wherein the distances from said pivots to the reference points are precisely known to within defined tolerances.

23. The system of claim 18, wherein in operation the camera or imaging device is held fixed at a defined position relative to the tracking device while the one or more images are being generated.

24. The system of claim 18, wherein the test object has a single reference point and is stepped throughout a defined workspace via a CMM.

26. The system of claim 20, wherein the defined workspace is a space associated with the camera or imaging system.