Method and system to assist 2D-3D image registration

• US 9,240,046 B2
• Filed: 03/01/2013
• Issued: 01/19/2016
• Est. Priority Date: 03/05/2012
• Status: Active Grant

• Current Assignee: CYDAR LIMITED
• Original Assignee: CYDAR LIMITED
• Inventors: Carrell, Tom, Varnavas, Andreas, Penney, Graeme
• Primary Examiner(s): Do, Anh
• Application Number:
  US14/382,999
• Publication Number:
  US 20150043798A1
• Time in Patent Office: 1,054 Days
• Field of Search: 382/285, 382/294, 382/154, 382/130, 382/131, 382/128, 382/151, 382/173, 382/190, 382/224, 382/239, 382/299, 382/300, 382/301, 358/1.2, 358/403, 345/472, 375/E7.088, 715/850
• US Class Current: 1/1
• CPC Class Codes: A61B 6/5223 generating planar views from image data, e.g. extracting a coronal view from a 3D image
• CPC Class Codes: G06T 2207/10064 Fluorescence image
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/30012 Spine; Backbone
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

Embodiments of the invention provide a system and method that is able to automatically provide a starting point for 2D to 3D image registration, without relying on human recognition of features shown in the 2D image. This is achieved by pre-processing the 3D data to obtain synthetically generated 2D images of those parts of the 3D data volume which will be used for registration purposes. Many different synthetically generated 2D images of the or each part of the 3D volume are produced, each from a different possible viewing direction. Each of these synthetic images is then subject to a feature extraction process to extract characterizing feature data of the registration feature shown in the images. Once the feature extraction has been undertaken for each image, when registration is to be performed the real-time 2D image is processed by applying each of the sets of extracted features thereto, to try and identify which set best matches the registration features in the 2D image. For example, where a generalized Hough transform was used in the feature extraction, the R tables would be applied to the 2D image to obtain respective accumulation images. The accumulation images may then be ranked to identify which registration feature is shown in the 2-D image, and from which view direction. This gives the required information of which registration feature is being shown in the 2D image, and also the in-plane location and orientation. This information can then be used as a starting point for the 2D to 3D registration procedure.

Claims

1. A method of determining a start position for a 2D to 3D image registration, the method comprising:

a) obtaining characteristic feature sets characterising one or more registration features imaged in a plurality of synthetic 2D images, the plurality of synthetic 2D images having been generated from a 3D image data set, the synthetic 2D images containing the one or more registration features having been imaged according to a plurality of respective viewing parameters

b) obtaining a 2D image to be registered with the 3D image data set;

c) applying the characteristic feature sets to the obtained 2D image to locate one or more registration features therein; and

d) determining which of the one or more characteristic features sets locate the one or more registration features in the obtained 2D image;

wherein at least the viewing parameters relating to the synthetic image corresponding to the determined characteristic features sets provide information relating to a start position for a subsequent registration of the obtained 2D image to the 3D image data set.

2. The method of claim 1, wherein the registration features are one or more vertebrae.

3. The method of claim 1, wherein the obtained 2D image is a fluoroscopy image.

4. The method of claim 1, wherein the 3D image data set is obtained via a computerized tomography (CT) scan.

5. The method of claim 1, wherein the characteristic feature sets are R tables for use with a generalized Hough transform, wherein the applying comprises generating accumulation images by applying the R-tables to the obtained 2D image using the generalized Hough transform.

6. The method of claim 5, wherein the determining comprises ranking the accumulation images to determine which R table best locates the registration feature in the obtained 2D image.

7. The method of claim 6, wherein the first N ranked accumulation images are further processed, the further processing comprising:

a) calculating a similarity value between the 3D image data set and the obtained 2D image for each of the N ranked accumulation images by means of a 2D-3D registration similarity measure; and

b) determining the start position for a subsequent 2D to 3D registration based on the accumulation image associated with the maximum calculated similarity value.

8. The method of claim 1, further comprising checking that two or more registration features are located in the obtained 2D image.

9. The method of claim 8, further comprising, for the two or more registration features, checking that the viewing parameters relating to the characteristic feature sets that located the registration features are within a predetermined distance of each other.

10. The method of claim 1, further comprising performing 2D to 3D image registration using the determined start position information.

11. The method of claim 10, further comprising checking that registration parameters achieved for different registration features are within a predetermined distance.

12. The method of claim 10, further comprising calculating a confidence value for the registration(s), and displaying the confidence value to the user.

13. A method of generating characteristic feature sets characterising a registration feature for use in the method of claim 1, the method comprising:

a) generating a plurality of synthetic 2D images from a 3D image data set, the synthetic 2D images containing one or more registration features imaged according to a plurality of respective viewing parameters;

b) generating characteristic feature sets from the synthetic 2D images characterising the registration features imaged therein; and

c) storing the generated characteristic feature sets.

14. The method of claim 13, wherein the registration features are one or more vertebrae.

15. The method of claim 13, wherein the 3D image data set is obtained via a computerised tomography (CT) scan.

16. The method of claim 13, wherein the characteristic feature sets are R tables for use with a generalised Hough transform.

20. The system of claim 19, wherein the one or more programs further causes the processor to generate characteristic feature sets characterising a registration feature for use in the method of claim 1, the method comprising:

(a) generating a plurality of synthetic 2D images from a 3D image data set, the synthetic 2D images containing one or more registration features imaged according to a plurality of respective viewing parameters;

(b) generating characteristic feature sets from the synthetic 2D images characterising the registration features imaged therein; and

(c) storing the generated characteristic feature sets;

and wherein the system includes one or more of features (i) to (iii), below;

(i) the registration features are one or more vertebrae;

(ii) the 3D image data set is obtained via a computerised tomography (CT) scan; and

(iii) the characteristic feature sets are R tables for use with a generalised Hough transform.

17. An image guided surgical system, comprising:

a 2D imaging system arranged in use to obtain a 2D image to be registered with a 3D image data set; and

a processor, arranged in use to;

a) obtain characteristic feature sets characterising one or more registration features imaged in a plurality of synthetic 2D images, the plurality of synthetic 2D images having been generated from the 3D image data set, the synthetic 2D images containing the one or more registration features having been imaged according to a plurality of respective viewing parameters;

b) apply the characteristic feature sets to the obtained 2D image to locate one or more registration features therein; and

c) determine which of the one or more characteristic features sets locate the one or more registration features in the obtained 2D image;

wherein the viewing parameters relating to the synthetic image corresponding to the determined characteristic feature sets provide information relating to a start position for registration of the obtained 2D image to the 3D image data set.

18. The system of claim 17, wherein the registration features are one or more vertebrae.

19. A system for generating characteristic feature sets characterising a registration feature for use with the system of claim 17, the system comprising:

a processor; and

a computer readable storage medium, the computer readable storage medium storing one or more programs so arranged such that when executed by the processor it/they cause the processor to;

a) generate a plurality of synthetic 2D images from a 3D image data set, the synthetic 2D images containing one or more registration features imaged according to a plurality of respective viewing parameters;

b) generate characteristic feature sets from the synthetic 2D images characterising the registration features imaged therein; and

c) store the generated characteristic feature sets.