Registration of fluoroscopic images of the chest and corresponding 3D image data based on the ribs and spine

• US 9,875,544 B2
• Filed: 08/07/2014
• Issued: 01/23/2018
• Est. Priority Date: 08/09/2013
• Status: Active Grant

• Current Assignee: Broncus Medical Incorporated
• Original Assignee: Broncus Medical Incorporated
• Inventors: Rai, Lav, Gibbs, Jason David, Wibowo, Henky
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US14/910,656
• Publication Number:
  US 20160180529A1
• Time in Patent Office: 1,265 Days
• Field of Search: 382128-134, 600407, 600410, 600411, 600427, 600594
• CPC Class Codes: G06T 2207/10121 Fluoroscopy
• CPC Class Codes: G06T 2207/30008 Bone
• CPC Class Codes: G06T 2207/30012 Spine; Backbone
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

The present invention is a method to register 3D image data with fluoroscopic images of the chest of a patient. The ribs and spine, which are visible in the fluoroscopic images, are analyzed and a rib signature or cost map is generated. The rib signature or cost map is matched to corresponding structures of the 3D image data of the patient. Registration is evaluated by computing a difference between the fluoroscopic image and a virtual fluoroscopic projected image of the 3D data. Related systems are also described.

Claims

1. A method to assist with registering 3D data with 2D fluoroscopic images of a thorax of a patient comprising the steps of:

receiving a 2D fluoroscopic image data set of the thorax including an anatomical structure which is visible under fluoroscopy; and

approximating a candidate location of the anatomical structure based on the 2D fluoroscopic image data set;

receiving a 3D data set of the anatomical structure in the thorax; and

registering the 3D data set with the 2D fluoroscopic image data set based on the approximating step;

wherein the 3D data set is at least one data selected from the group comprising image data and model data; and

wherein the registering step comprises computing a 3D-projected location of the anatomical structure from the 3D data set, and matching the 3D-projected location of the anatomical structure from the 3D data set with the candidate location of the anatomical structure from the approximating step.

2. The method of claim 1, wherein the anatomical structure is one structure selected from the group comprising a rib and spine.

3. The method of claim 1, further comprising estimating a motion between the candidate location of the anatomical structure from the approximating step, and the 3D-projected location of the anatomical structure from the 3D data set.

4. The method of claim 1, further comprising, at least in part, manually selecting the candidate location of the anatomical structure in the 2D fluoroscopic image data set.

5. The method of claim 1, wherein the approximating step comprises automatically determining a probability map of rib and spine locations.

6. The method of claim 5, wherein the probability is generated by applying a filter which assigns a high cost for local stripes.

7. The method of claim 1, further comprising computing centerlines of the anatomical structure from the 3D data set, and performing the registration step based on the centerlines.

8. A method for evaluating a completed registration between a 2D fluoro image of the thorax and a 3D-based fluoro projection image computed from a 3D data set, the method comprising:

estimating a 2D displacement motion between the 2D fluoro image and the 3D-based fluoro projection image; and

computing a final displacement error based on the 2D displacement motion from the estimating step; and

wherein the 2D displacement motion comprises a plurality of 2D displacement values, and the final displacement error is based on weighting at least one displacement value more than another.

9. The method of claim 8, wherein the step of estimating the 2D displacement motion is performed with a block matching algorithm.

10. The method of claim 8, wherein higher weighting is assigned to displacement values associated with a boney structure.

11. The method of claim 10, wherein the boney structure comprises ribs.

12. The method of claim 11, wherein the boney structure excludes the diaphragm.

13. The method of claim 12, wherein, prior to computing the displacement error, a rib-probability map is generated based on the 2D fluoro image of the thorax and approximating a candidate location of the ribs.

14. The method of claim 8, further comprising detecting patient motion post-registration by:

receiving a current fluoroscopic image;

computing a current 2D displacement motion between the current fluoroscopic image and the 3D-based fluoro projection image.

15. The method of claim 14, further comprising indicating whether the patient has moved.

16. The method of claim 15, wherein the step of indicating is performed by computing whether the current 2D displacement motion exceeds an acceptable range.

17. A fluoroscopy assistance system to assist with registering 3D data with 2D fluoroscopic images of a thorax of a patient comprising a computer processor programmed to:

receive a 2D fluoroscopic image data set of the thorax including an anatomical structure which is visible under fluoroscopy;

receive a 3D data set of the thorax including the anatomical structure;

computing a candidate signature representing the anatomical structure based on the 2D fluoroscopic image data and a virtual signature based on the 3D data set;

register the 3D data set with the 2D fluoroscopic image data set based on the at least one of the candidate signature and the virtual signature; and

computing a difference between the candidate signature of the anatomical structure and the virtual signature of the anatomical structure from the 3D data set

wherein the anatomical structure is one structure selected from the group comprising a rib and spine; and

wherein the 3D data set is at least one data selected from the group comprising image data and model data.

18. The system of claim 17, wherein the virtual signature is a virtual fluoroscopic projected image of the anatomical structure.

19. The system of claim 17, wherein the virtual signature comprises centerlines of ribs.

20. The system of claim 17, comprising computing the candidate signature and the virtual signature wherein the candidate signature is based on determining a probability map of rib and spine locations.

21. The system of claim 20, further comprising computing alignment cost between the virtual signature and the candidate signature.

22. The system of claim 21, wherein the match is based on the maximizing the alignment cost.

23. The system of claim 17, comprising computing the candidate signature and the virtual signature, and matching the candidate signature to the virtual signature.