COMPOSITE IMAGES FOR MEDICAL PROCEDURES

• US 20110262016A1
• Filed: 12/05/2008
• Published: 10/27/2011
• Est. Priority Date: 12/07/2007
• Status: Active Grant

• Current Assignee: University of Maryland Baltimore County (University of Maryland)
• Original Assignee: University of Maryland Baltimore County (University of Maryland)
• Inventors: Shekhar, Raj, Dandekar, Omkar
• Granted Patent: US 8,207,992 B2
• Time in Patent Office: Days
• US Class Current: 382/131
• CPC Class Codes: A61B 6/037 Emission tomography
• CPC Class Codes: A61B 6/5247 combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound
• CPC Class Codes: G06T 2200/28 involving image processing hardware
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/10104 Positron emission tomography [PET]
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: G06T 7/35 using statistical methods
• CPC Class Codes: G06T 7/38 Registration of image sequences

Abstract

Medical imaging often involves the collective use of information presented in multiple images of an individual, such as images generated through different imaging modalities (X-ray, CT, PET, etc.) The use of a composite of these images may involve image registration to adjust for the variable position and orientation discrepancies of the individual during imaging. However, registration may be complicated by soft tissue deformation between images, and implementations (particularly pure software implementations) of the mathematical models used in image registration may be computationally complex and may require up to several hours. Hardware architectures are presented that apply the mathematical techniques in an accelerated manner, thereby providing near-realtime image registration that may be of particular use for the short timeframe requirements of surgical environments. The composite image generated thereby may be used to target anatomic features of interest during various medical procedures, including surgical procedures. Moreover, such techniques may be applied to computationally difficult image processing techniques, such as the display of a composite image based at least in part on a PET image, which may otherwise be difficult to utilize in a time-sensitive manner such as a surgical setting.

Claims

1. A system for registering a floating image with a reference image, the system comprising:

a reference image memory configured to store a reference image voxel set representing the reference image;

a floating image memory configured to store a floating image voxel set the floating image;

a voxel enumerator unit configured to count the voxels in sequential order along an axis of the reference image voxel set;

a voxel coordinate transformation unit configured to transform the voxels of the floating image voxel set with respect to the voxel in the reference image voxel set indicated by the voxel enumerator unit and to produce a transformed floating image voxel set;

a mutual histogram accumulator unit configured to accumulate a mutual histogram representing the transformed floating image voxel set interpolated with the reference image voxel set;

a volumetric interpolator unit configured to store in the mutual histogram accumulator unit the contribution to the mutual histogram of respective voxels of the transformed floating image voxel set interpolated with respective voxels of the reference image voxel set; and

a mutual information calculator unit configured to calculate the mutual information between the reference image voxel set and the transformed floating image voxel set based on the mutual histogram.

2. The system of claim 1, wherein the floating image is obtained utilizing a preoperative PET imaging modality.

3. The system of claim 1, wherein the reference image is obtained utilizing one of a preoperative PET imaging modality, an X-ray imaging modality, an X-ray CT imaging modality, an ultrasound imaging modality, and an MRI imaging modality.

4. The system of claim 1, wherein the floating image memory configured comprises a configuration to store at least four copies of the floating image voxel set and to read a voxel octet from the floating image voxel set corresponding to a voxel in the reference image voxel set.

5. The system of claim 1, wherein the mutual information calculator unit comprises a configuration to calculate the mutual information based on the joint probabilities between the reference image voxel set and the transformed floating image voxel set as an aggregate of the individual probabilities of the reference image voxels with respective transformed floating image voxels.

6. The system of claim 5, wherein the configuration of the mutual information calculator unit further comprises a logarithmic lookup table, and the mutual information calculator unit is configured to consult the logarithmic lookup table to compute logarithmic components of the individual probabilities.

7. The system of claim 1, comprising an image registration control unit comprising a configuration to perform multi-pass registration of the floating image with the reference image until the mutual information between the reference image voxel set and the transformed floating image voxel set is within a desired mutual information range.

8. The system of claim 7, wherein the image registration control unit is configured, before initiating an at least second registration pass, to subdivide the voxels of the transformed floating image voxel set.

9. The system of claim 1, comprising a field-programmable gate array comprising a configuration to represent at least one of the voxel enumerator unit, voxel coordinate transformation unit, the mutual histogram accumulator unit, the volumetric interpolator unit, and the mutual information calculator unit.

10. A system for displaying a composite image from a floating image and a reference image, the system comprising:

a reference image memory configured to store a reference image voxel set representing the reference image;

a floating image memory configured to store a floating image voxel set the floating image;

a voxel enumerator unit configured to count the voxels in sequential order along an axis of the reference image voxel set;

a voxel coordinate transformation unit configured to transform the voxels of the floating image voxel set with respect to the voxel in the reference image voxel set indicated by the voxel enumerator unit and to produce a transformed floating image voxel set;

a mutual histogram accumulator unit configured to accumulate a mutual histogram representing the transformed floating image voxel set interpolated with the reference image voxel set;

a volumetric interpolator unit configured to store in the mutual histogram accumulator unit the contribution to the mutual histogram of respective voxels of the transformed floating image voxel set interpolated with respective voxels of the reference image voxel set;

a mutual information calculator unit configured to calculate the mutual information between the reference image voxel set and the transformed floating image voxel set based on the mutual histogram;

an image registration control unit configured to perform multi-pass registration of the floating image with the reference image until the mutual information between the reference image voxel set and the transformed floating image voxel set is within a desired mutual information range;

a composite image generating component configured to generate a composite image from the reference image voxel set and the transformed floating image voxel set; and

a display unit configured to display the composite image.

11. The system of claim 10, the floating image utilizing a preoperative PET imaging modality.

12. The system of claim 10, the reference image utilizing at least one of a preoperative PET imaging modality, an X-ray imaging modality, an X-ray CT imaging modality, an ultrasound imaging modality, and an MRI imaging modality.

13. The system of claim 10, comprising an imaging unit configured to generate at least one of the reference image and the floating image.

14. The system of claim 10, comprising a field-programmable gate array configured to represent at least one of the voxel enumerator unit, voxel coordinate transformation unit, the mutual histogram accumulator unit, the volumetric interpolator unit, and the mutual information calculator unit.

15. A method of targeting an anatomic feature of an individual pertaining to a medical procedure illustrated in a reference image of the individual utilizing a first imaging modality and a floating image of the individual utilizing a floating imaging modality, the method comprising:

generating a composite image from the reference image and the floating image; and

targeting the anatomic feature using the composite image in furtherance of the medical procedure.

16. The method of claim 15, wherein said targeting comprises evaluating at least one of the presence, the location, the size, and the condition of the anatomic feature in relation to the health state of the individual.

17. The method of claim 15, wherein said medical procedure comprises a surgical procedure, and the targeting comprises using the composite image to guide the identification and localization of the anatomic feature in a surgical procedure.

18. The method of claim 17, wherein using the composite image to guide comprises:

detecting the position of a surgical probe within the surgical site, and

displaying the position of the surgical probe coordinated with the composite image.

19. The method of claim 15, comprising generating at least one of the reference image and the floating image with an imaging component.

20. The method of claim 19, wherein the medical procedure comprising a surgical procedure comprises at least one of the reference image and the floating image generated intraoperatively.

21. The method of claim 15, wherein at least one of the reference imaging modality and the floating imaging modality comprises a PET imaging modality.

22. The method of claim 15, wherein generating a composite image comprises applying a system of generating a composite image to the reference image and the floating image.

23. The method of claim 22, wherein the system comprises a configuration to generate the composite image in at least a near-realtime performance.

24. The method of claim 23, wherein the system comprises:

a reference image memory configured to store a reference image voxel set representing the reference image;

a floating image memory configured to store a floating image voxel set the floating image;

a voxel enumerator unit configured to count the voxels in sequential order along an axis of the reference image voxel set;

a voxel coordinate transformation unit configured to transform the voxels of the floating image voxel set with respect to the voxel in the reference image voxel set indicated by the voxel enumerator unit and to produce a transformed floating image voxel set;

a mutual histogram accumulator unit configured to accumulate a mutual histogram representing the transformed floating image voxel set interpolated with the reference image voxel set;

a volumetric interpolator unit configured to store in the mutual histogram accumulator unit the contribution to the mutual histogram of respective voxels of the transformed floating image voxel set interpolated with respective voxels of the reference image voxel set;

a mutual information calculator unit configured to calculate the mutual information between the reference image voxel set and the transformed floating image voxel set based on the mutual histogram;

an image registration control unit configured to perform multi-pass registration of the floating image with the reference image until the mutual information between the reference image voxel set and the transformed floating image voxel set is within a desired mutual information range; and

a composite image generating component configured to generate a composite image from the reference image voxel set and the transformed floating image voxel set.

25. The method of claim 24, wherein the system comprises a field-programmable gate array comprising a configuration to represent at least one of the voxel enumerator unit, voxel coordinate transformation unit, the mutual histogram accumulator unit, the volumetric interpolator unit, and the mutual information calculator unit.