Method and apparatus for performing 2D to 3D registration

• US 7,570,791 B2
• Filed: 08/20/2003
• Issued: 08/04/2009
• Est. Priority Date: 04/25/2003
• Status: Active Grant

• Current Assignee: Medtronic Navigation Incorporated (Medtronic Plc)
• Original Assignee: Medtronic Navigation Incorporated (Medtronic Plc)
• Inventors: Frank, Kevin J, Mark, Kevin E
• Primary Examiner(s): Mehta, Bhavesh M
• Assistant Examiner(s): MOTSINGER, SEAN T
• Application Number:
  US10/644,680
• Publication Number:
  US 20040215071A1
• Time in Patent Office: 2,176 Days
• Field of Search: 382/285, 128/920, 600/407
• US Class Current: 382/132
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 6/4441 the rigid structure being a C-arm or U-arm
• CPC Class Codes: A61B 6/463 characterised by displaying multiple images or images and diagnostic data on one display
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for

Abstract

A method and apparatus for performing 2D to 3D registration includes an initialization step and a refinement step. The initialization step is directed to identifying an orientation and a position by knowing orientation information where data images are captured and by identifying centers of relevant bodies. The refinement step uses normalized mutual information and pattern intensity algorithms to register the 2D image to the 3D volume.

Claims

1. A method for registering two-dimensional image data with three-dimensional image data of a body of interest, said method comprising:

acquiring the three-dimensional image data having first patient orientation information;

acquiring the two-dimensional image data having second patient orientation information; and

generating a digitally reconstructed registered radiograph that substantially corresponds to the two-dimensional image data using the three-dimensional image data and the first and second patient orientation information, including;

(a) wherein the first patient orientation information includes determining how the patient was positioned or oriented relative to an imaging device during acquiring the three-dimensional image data and wherein the second patient orientation includes determining how the patient was positioned or oriented relative to an imaging device during acquiring the two-dimensional image data;

(b) determining an estimate of the patient'"'"'s orientation with respect to a dynamic reference frame based at least in part on (a);

(c) wherein the digitally reconstructed radiograph includes a view through the three-dimensional image data, along the direction of the two-dimensional image data based upon the determined first and second patient orientation;

refining the registration of the two-dimensional image data with the three-dimensional image data using the first and second patient orientation information and the common point information;

wherein the refined registration employs at least two similarity/cost measures selected from a group of at least a normalized mutual information algorithm, a mutual information algorithm, a gradient difference algorithm, a gradient algorithm, a line contour algorithm, a surface contour algorithm, a pattern intensity algorithm or a combination thereof.

2. The method as defined in claim 1 wherein acquiring two-dimensional image data further includes acquiring a two-dimensional anterior to posterior image and a two-dimensional lateral image.

3. The method as defined in claim 2 further comprising identifying a center of a body of interest in the two-dimensional anterior to posterior image and the two-dimensional lateral image.

4. The method as defined in claim 3 wherein generating a digitally reconstructed radiograph further includes generating an anterior to posterior digitally reconstructed radiograph and a lateral digitally reconstructed radiograph corresponding to the two-dimensional anterior to posterior image and the two-dimensional lateral image.

5. The method as defined in claim 4 further comprising identifying a center of the body of interest in the anterior to posterior digitally reconstructed radiograph and the lateral digitally reconstructed radiograph.

6. The method as defined in claim 5 further comprising identifying a common point in the three-dimensional image data with the two-dimensional image data using the identified centers of the anterior to posterior image, lateral image, anterior to posterior digitally reconstructed radiograph image and lateral digitally reconstructed radiograph image.

7. The method as defined in claim 6 further comprising refining the registration of the two-dimensional image data with the three-dimensional image data using the first and second patient orientation information and the common point information.

8. The method as defined in claim 1 further comprising optimizing the selected similarity/cost measures using an optimization algorithm selected from a group of at least a multi-stage steepest ascent algorithm, a steepest ascent algorithm, a gradient-based optimization algorithm or a combination thereof.

9. The method as defined in claim 7 further comprising adjusting a registration window on the body of interest in each anterior to posterior image, lateral image, anterior to posterior digitally reconstructed radiograph image, and lateral digitally recon radiograph image, wherein only the image data within the registration windows are used for refined registration.

10. The method as defined in claim 8 further comprising verifying the refined registration for accuracy by selecting a point in the three-dimensional image data to confirm its accuracy with a point in the two-dimensional image data.

11. The method as defined in claim 1 further comprising performing a refinement registration utilizing normalized mutual information and pattern intensity.

12. The method as defined in claim 11 further comprising optimizing the refinement registration by utilizing a multi-stage steepest ascent algorithm.

13. The method as defined in claim 1 further comprising performing multiple registrations on multiple bodies of interest.

14. The method as defined in claim 1 wherein the body of interest is a vertebrae.

15. A method for registering two-dimensional image data with three-dimensional image data of a body of interest, said method comprising:

acquiring the three-dimensional image data;

acquiring the two-dimensional image data;

generating a digitally reconstructed radiograph using the three-dimensional image data; and

registering the two-dimensional image data with the three-dimensional image data using both a first similarity/cost measure and a second similarity/cost measure; and

displaying a single registered image based upon the registration of the two-dimensional image data with the three-dimensional image data on a display;

wherein registering the two-dimensional image data with the three-dimensional image data includes determining a transformation to correlate points between the two-dimensional image data and the three-dimensional image data;

wherein both the first similarity/cost measure and the second similarity/cost measure are optimized to register the two-dimensional image data with the three-dimensional image data.

16. The method as defined in claim 15 further comprising optimizing the first similarity/cost measure and the second similarity/cost measure.

17. The method as defined in claim 15 wherein the first similarity/cost measure is normalized mutual information and the second similarity/cost measure is pattern intensity.

18. The method as defined in claim 17 further comprising optimizing the normalized mutual information and the pattern intensity utilizing a multi-stage steepest ascent algorithm.

19. The method as defined in claim 15 further comprising verifying registration accuracy including displaying a graph of the registration process illustrating the convergence of the process to an answer.

20. The method as defined in claim 15 further comprising adjusting a registration window in the two-dimensional image data and a registration window in the digitally reconstructed radiograph where the image data within the registration windows are used for registration.

21. The method as defined in claim 15 wherein acquiring three-dimensional image data further includes acquiring three-dimensional image data having first patient orientation information and wherein acquiring two-dimensional image data further includes acquiring two-dimensional image data having second patient orientation information and wherein generating the digitally reconstructed radiograph further includes using the first and second patient orientation information.

22. The method as defined in claim 15 further comprising performing intensity adjustment on the two-dimensional image data.

23. The method as defined in claim 22 wherein performing intensity adjustment on the two-dimensional image data includes performing intensity adjustment on a two-dimensional lateral image of the body of interest, wherein generating a digitally reconstructed radiograph further includes generating a lateral digitally reconstructed radiograph that substantially corresponds to the two-dimensional lateral image.

24. The method as defined in claim 23 further comprising aligning the two-dimensional lateral image with the lateral digitally reconstructed radiograph utilizing at least one similarity/cost measure.

39. The method as defined in claim 15 further comprising verifying registration accuracy including:

selecting a point in the acquired three-dimensional image data and representing the selected point on a display relative to the acquired two-dimensional image data;

determining whether the selected point is in a correct position in the two-dimensional image data.

25. A method for registering two-dimensional image data with three-dimensional image data of a body of interest, said method comprising:

acquiring the three-dimensional image data of the body of interest;

acquiring a two-dimensional lateral image of the body of interest;

generating a digitally reconstructed radiograph that substantially corresponds to the two-dimensional image;

performing intensity adjustment of the two-dimensional image to reduce the effect of an interfering object; and

aligning the two-dimensional image with the digitally reconstructed radiograph using a similarity/cost measure;

comprising acquiring a two-dimensional anterior to posterior image and generating an anterior to posterior digitally reconstructed radiograph that substantially corresponds to the anterior to posterior image;

registering the two-dimensional lateral image and the two-dimensional anterior to posterior image with the three-dimensional image data using a first similarity/cost measure and a second similarity/cost measure.

26. The method as defined in claim 25 wherein acquiring the three-dimensional image data further includes acquiring first patient orientation information, wherein acquiring the two-dimensional lateral image and the two-dimensional anterior to posterior image further includes acquiring second patient orientation information, and wherein generating the digitally reconstructed radiographs further includes generating the lateral digitally reconstructed radiograph and the anterior to posterior digitally reconstructed radiograph using the three-dimensional image data and the first and second patient orientation information.

27. The method as defined in claim 26 further comprising identifying a center of the body of interest in the two-dimensional anterior to posterior image and the two-dimensional lateral image.

28. The method as defined in claim 27 further comprising identifying a center of the body of interest in the anterior to posterior digitally reconstructed radiograph and the lateral digitally reconstructed radiograph.

29. The method as defined in claim 28 further comprising identifying a common point in the three-dimensional image data with the two-dimensional image data using the identified centers of the anterior to posterior image, lateral image, anterior to posterior digitally reconstructed radiograph image and lateral digitally reconstructed radiograph image.

30. The method as defined in claim 25 further comprising optimizing the first and second similarity/cost measures using an optimization algorithm.

37. The method of claim 25, wherein performing intensity adjustment of the two-dimensional image to reduce the effect of an interfering object includes:

determining a histogram of intensity of the two-dimensional image;

determining a mean of the intensity; and

adjusting the mean of the intensity to a target value.

38. The method of claim 37, wherein adjusting the mean to a target value includes adjusting the contrast, the brightness, the level, or the width, or combinations thereof to modify the determined mean.

31. A method for registering two-dimensional image data with three-dimensional image data of a body of interest, said method comprising:

acquiring the three-dimensional image data having first patient orientation information;

acquiring a first two-dimensional image having second patient orientation information;

acquiring a second two-dimensional image having third patient orientation information;

identifying a center of the body of interest in the first and second two-dimensional images;

generating first and second digitally reconstructed radiographs;

identifying the center of the body of interest in the first and second digitally reconstructed radiographs; and

registering the first and second two-dimensional images with the three-dimensional image data using at least both a first similarity/cost measure and a second similarity/cost measure.

32. The method as defined in claim 31 wherein said first two-dimensional image is a first two-dimensional anterior to posterior image and said second two-dimensional image is a second two-dimensional lateral image.

33. The method as defined in claim 32 further comprising performing intensity adjustment on the two-dimensional lateral image and aligning the two-dimensional lateral image with the lateral digitally reconstructed radiograph.

34. The method as defined in claim 31 wherein the first similarity/cost measure is normalized mutual information and the second similarity/cost measure is pattern intensity.

35. The method as defined in claim 31 further comprising optimizing the first and second similarity/cost measures.

36. The method as defined in claim 35 wherein first and second similarity/cost measures are optimized using a multi-stage steepest ascent algorithm.

40. The method of claim 31, wherein identifying a center of the body of interest includes moving a symbol overlaid on at least one of the first and second two-dimensional images to a center of a vertebral body.

41. The method of claim 40, further comprising:

providing an atlas map of the vertebral body;

using the atlas map to provide at least an initial estimate for the center of the vertebral body.

42. A method for registering two-dimensional image data with three-dimensional image data of a vertebral body of interest, said method comprising:

acquiring the three-dimensional image data having first patient orientation information;

acquiring a first two-dimensional image having second patient orientation information;

acquiring a second two-dimensional image having third patient orientation information;

displaying the first and second two-dimensional images;

identifying a center of the vertebral body of interest with the displayed first and second two-dimensional images;

generating first and second digitally reconstructed radiographs;

aligning the first and second two-dimensional image with the respective digitally reconstructed radiographs;

identifying the center of the body of interest in the first and second digitally reconstructed radiographs;

selecting a first similarity/cost measure and a second similarity/cost measure; and

after aligning the first and second two-dimensional images with the respective digitally reconstructed radiographs, registering the first and second two-dimensional images with the three-dimensional image data using at least both the first similarity/cost measure and the second similarity/cost measure, including optimizing both the first similarity/cost measure and the second similarity/cost measure.

43. The method as defined in claim 42 further comprising performing intensity adjustment on the two-dimensional image.

44. The method of claim 42, further comprising:

acquiring the first and second two-dimensional images during an operative procedure and after acquiring the three-dimensional image data.

45. The method of claim 42, wherein aligning the first and second two-dimensional images with the respective digitally reconstructed radiographs includes:

determining a region of interest in at least one of the first and second two-dimensional images with the respective digitally reconstructed radiographs;

selecting a third similarity/cost measure;

executing a set of instructions to align the at least one of the first and second two-dimensional images with the respective digitally reconstructed radiographs based upon the selected third similarity/cost measure.

46. A method for registering two-dimensional image data with three-dimensional image data of a body of interest, said method comprising:

acquiring the three-dimensional image data having first patient orientation information;

acquiring the two-dimensional image data having second patient orientation information; and

generating a digitally reconstructed radiograph that substantially corresponds to the two-dimensional image data using the three-dimensional image data and the first and second patient information, including;

(a) wherein the first patient information includes determining how the patient was positioned or oriented relative to an imaging device during acquiring the three-dimensional image data and wherein the second patient information includes determining how the patient was positioned or oriented relative to an imaging device during acquiring the two-dimensional image data;

(b) determining an estimate of the patient'"'"'s orientation with respect to a dynamic reference frame based at least in part on (a);

(c) wherein the digitally reconstructed radiograph includes a view through the three-dimensional image data, along the direction of the two-dimensional image data based upon the determined first and second patient orientation;

wherein the two-dimensional image data is acquired intraoperatively;

wherein acquiring two-dimensional image data further includes acquiring a two-dimensional anterior to posterior image and a two-dimensional lateral image and identifying a center of a body of interest in the two-dimensional anterior to posterior image and the two-dimensional lateral image;

wherein generating a digitally reconstructed radiograph further includes generating an anterior to posterior digitally reconstructed radiograph and a lateral digitally reconstructed radiograph corresponding to the two-dimensional anterior to posterior image and the two-dimensional lateral image and identifying a center of the body of interest in the anterior to posterior digitally reconstructed radiograph and the lateral digitally reconstructed radiograph;

identifying a common point in the three-dimensional image data with the two-dimensional image data using the identified centers of the anterior to posterior image, lateral image, anterior to posterior digitally reconstructed radiograph image and lateral digitally reconstructed radiograph image;

wherein determining the estimate of the patient'"'"'s orientation with respect to the dynamic reference frame is determined by combining the first patient information and the second patient information including the information of where the patient was positioned or oriented for the acquisition of the two-dimensional image data and how the patient was oriented or positioned during the acquisition of the three-dimensional image data;

wherein the estimate of the patient'"'"'s orientation with respect to the dynamic reference frame is used to generate the digitally reconstructed radiographs to correspond substantially to the two-dimensional intraoperative image data.