ROI selection in image registration

• US 7,231,076 B2
• Filed: 06/30/2004
• Issued: 06/12/2007
• Est. Priority Date: 06/30/2004
• Status: Active Grant

• Current Assignee: Accuray Incorporated
• Original Assignee: Accuray Incorporated
• Inventors: Kuduvalli, Gopinath, Fu, Dongshan
• Primary Examiner(s): Johns; Andrew W.
• Assistant Examiner(s): MACKOWEY, ANTHONY M
• Application Number:
  US10/881,209
• Publication Number:
  US 20060002631A1
• Time in Patent Office: 1,077 Days
• Field of Search: 382/107, 382128-132, 382294-298, 378 4- 27, 378/62, 378/63, 378/65
• US Class Current: 382/131
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/10124 Digitally reconstructed radiograph [DRR]
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/238 using non-full search, e.g. three-step search
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]

Abstract

A method and system is presented for automatically selecting a region of interest (ROI) within an image of an object, for example when performing non-rigid image registration between the image and another image of the object. In this way, the need for user interaction can be minimized or eliminated during image registration. The ROI is determined by defining an entropy measure H of the image, and selecting the region within the image in which the entropy measure is maximized, In this way, the ROI is optimized to contain as much information as possible. In one embodiment, the entropy measure H is a modified Shannon entropy, defined by H=−ΣI P(I) log P(I), where I is the value of the image intensity level, and P(I) is the probability of an image intensity value I occurring within the ROI.

Claims

1. An image registration system for registering at least one 2D image of an anatomical region with previously generated 3D scan data of the anatomical region, the anatomical region including at least one treatment target and at least one reference structure, wherein the 2D image is generated in near real time by detecting one or more radiographic imaging beams after the imaging beams have traversed at least a portion of the anatomical region, the imaging beams having known intensities and known positions and angles relative to the anatomical region, the system comprising:

means for providing the 3D scan data of the anatomical region;

a scan data modifier configured to modify the 3D scan data so as to compensate for a difference between the ratio of bone-to-tissue attenuation at the energy level of the 3D scan, and the ratio of bone-to-tissue attenuation at the energy level of the imaging beam used for the near real-time 2D image;

a DRR generator configured to generate at least one DRR (digitally reconstructed radiograph) of the anatomical region, using the 3D scan data and the known locations, angles, and intensities of the imaging beams;

an ROI selector configured to select an ROI (region of interest) within the DRR, the ROI including the image of the treatment target and of the reference structure;

a motion field generator configured to generate a 3D full motion field within the ROI by estimating a plurality of local motion fields within the ROI; and

a parameter determiner configured to determine from the 3D full motion field a set of non-rigid transformation parameters that represent the difference in the position and orientation of the treatment target as shown in the 2D image, as compared to the position and orientation of the treatment target as shown in the DRR.

2. A system in accordance with claim 1,

wherein the 3D scan data comprise a plurality of CT numbers representing the image intensity of corresponding 3D CT voxels, each CT voxel representing a corresponding volume element of the anatomical region;

wherein each CT voxel is disposed within one of a plurality of axial voxel slices, each axial voxel slice representing a corresponding axial slice of the anatomical region;

wherein each CT number represents the attenuated intensity of an x-ray CT beam that has been generated at a CT scan energy level and that has traversed the corresponding volume element of the anatomical region.

3. A system in accordance with claim 2,

wherein the scan data modifier includes a processor for performing on each CT number a mathematical operation derived from a non-linear x-ray attenuation model;

and

wherein the mathematical operation is given by;

C(x,y,z)=a C₀(x,y,z) e^bC0(x,y,z)

where C(x,y,z) represents the modified CT number of a 3D CT voxel having a location (x,y,z);

a and b represent weighting coefficients;

and C₀(x,y,z) represents the un-modified CT number, based on a linear attenuation model, of a 3D CT voxel having a location (x,y,z).

4. A system in accordance with claim 1, wherein the motion field generator comprises:

a mesh grid generator configured to generate in the 2D image a mesh grid having a plurality of mesh nodes;

a nodal motion estimator configured to estimate, for each mesh node in the 2D image, at least one nodal motion vector that describes a matching of the mesh node with a corresponding mesh node in the DRR; and

a motion field interpolator configured to determine a local motion vector for each of a plurality of points of interest within the first image, by interpolating from the nodal motion vectors of the mesh nodes that surround each point of interest.

5. A system in accordance with claim 4,

wherein the mesh grid generator is configured to repeat, at each of a plurality of mesh resolution levels, the act of generating of the mesh grid;

wherein nodal motion estimator is configured to repeat, at each of the plurality of mesh resolution levels, the act of estimating the at least one nodal motion vector for each mesh node; and

wherein the nodal motion vectors from which the motion field interpolator interpolates to determine the local motion vector comprise nodal motion vectors that have been estimated at a final one of the plurality of mesh resolution levels.

6. A system in accordance with claim 5,

further comprising a motion field reconstructor configured to reconstruct the nodal motion vector for a mesh node if any nodal motion vector estimated for any mesh node is unreliable.

7. A system in accordance with claim 5,

wherein at each mesh resolution level, the nodal motion estimator is configured to pass onto a subsequent mesh resolution level one or more nodal motion vectors that have been estimated at the current mesh resolution level.

8. A system in accordance with claim 5, wherein the plurality of mesh levels comprise successively increasing mesh resolution levels, the mesh grid at each successive mesh resolution level having a number of mesh nodes that is greater compared to the number of mesh nodes at each previous mesh resolution level.

9. A system in accordance with claim 8,

wherein for the first one of the plurality of mesh resolution levels, the nodal motion estimator is configured to estimate a global translation for the 2D image and to use the global translation as an estimate for the nodal motion vector for each mesh node in the mesh grid generated for the first mesh resolution level;

and

wherein the global translation represents a translation of an image center of the 2D image.

10. A system in accordance with claim 5,

wherein at each mesh resolution level subsequent to the first mesh resolution level, the mesh nodes in the 2D image comprise a first subset of mesh nodes and a second subset of mesh nodes;

wherein the first subset of mesh nodes comprise mesh nodes generated at a previous mesh resolution level; and

wherein the second subset of mesh nodes comprise mesh nodes that are newly added at the current mesh resolution level.

11. A system in accordance with claim 10,

wherein at each mesh resolution level subsequent to the first mesh resolution level, the nodal motion estimator is configured to use the nodal motion vectors passed on from a previous mesh resolution level as estimates for nodal motion vectors for the first subset of mesh nodes;

wherein the nodal motion estimator comprises an interpolator configured to interpolate from the nodal motion vectors estimated for the first subset of mesh nodes, to estimate the nodal motion vectors for the second subset of mesh nodes; and

wherein the nodal motion estimator further comprises a nodal motion refiner configured to refine, for each mesh node in both the first and the second subsets, the nodal motion vector that was estimated for the mesh node.

12. A system in accordance with claim 11,

wherein the nodal vector refiner is configured to:

define a block centered on a mesh node in the 2D image;

search for a matching mesh node in the DRR that maximizes a similarity measure between the block in the 2D image and another block centered around the matching mesh node in the DRR; and

modify the nodal motion vector so that the nodal motion vector describes a mapping of the mesh node in the 2D image onto the matching mesh node in the DRR.

13. A system in accordance with claim 12,

wherein the nodal vector refiner is configured to repeat, for each of a plurality of successively increasing image resolution levels, the acts of defining a block centered around a mesh node in the 2D image, searching for a matching mesh node in the DRR, and refining the nodal motion vector.

14. A method of registering a near real time 2D x-ray image of an anatomical region with 3D scan data representative of a preoperative image of the anatomical region, the anatomical region including at least one reference structure and at least one treatment target, the method comprising:

modifying the 3D scan data, and reconstructing from the modified 3D scan data at least one DRR;

selecting an ROI (region of interest) within the DRR, wherein the ROI includes at least one structure in the object;

generating a 3D motion field by estimating one or more 2D local motion fields within the ROI, and constructing a full 3D motion field from the local motion fields; and

determining from the full 3D motion field a set of non-rigid transformation parameters that represent the difference in the position and orientation of the reference structure and the treatment target as shown in the 2D x-ray image, as compared to the position and orientation of the reference structure and the treatment target as shown in the DRR.

15. A method in accordance with claim 14,

wherein the anatomical region comprises bone and tissue;

wherein the 3D scan data are modified to compensate for a difference between the ratio of bone-to-tissue attenuation at the energy level of the 3D scan, and the ratio of bone-to-tissue attenuation at the energy level of the x-ray imaging beam used for the near real-time x-ray image.

16. A method in accordance with claim 15,

wherein the 3D scan data comprise a plurality of CT numbers representing the image intensity of corresponding 3D CT voxels, each CT voxel representing a corresponding volume element of the anatomical region;

wherein each CT voxel is disposed within one of a plurality of axial voxel slices, each axial voxel slice representing a corresponding axial slice of the anatomical region;

wherein each CT number represents the attenuated intensity of an x-ray CT beam that has been generated at a CT scan energy level and that has traversed the corresponding volume element of the anatomical region.

17. A method in accordance with claim 16,

wherein the scan data modifier includes a processor for performing on each CT number a mathematical operation derived from a non-linear x-ray attenuation model;

and

wherein the mathematical operation is given by;

C(x,y,z)=a C₀(x,y,z) e^bC0(x,y,z)

where C(x,y,z) represents the modified CT number of a 3D CT voxel having a location (x,y,z);

a and b represent weighting coefficients;

and C₀(x,y,z) represents the un-modified CT number, based on a linear attenuation model, of a 3D CT voxel having a location (x,y,z).

18. A method in accordance with claim 14, wherein the step of defining the ROI comprises:

a) calculating an entropy measure H of the DRR; and

b) selecting an ROI so as to maximize the entropy measure H.

19. A method in accordance with claim 18, wherein the entropy measure H is a modified Shannon entropy measure given by:

H=−

Σ

_I=1^I=Imax I P(I) log P(I),

where

I is the value of the intensity of the first image, at each pixel of the image;

I_max is the number of image intensity levels; and

P(I) is the probability of an image intensity value I occurring within the ROI.

20. A method in accordance with claim 14, wherein the act of generating a motion field comprises:

generating in the 2D x-ray image a mesh grid having a plurality of mesh nodes;

for each mesh node in the 2D x-ray image, estimating at least one nodal motion vector that describes a matching of the mesh node with a corresponding mesh node in the DRR; and

determining a local motion vector for each of a plurality of points of interest within the 2D x-ray image, by interpolating from the nodal motion vectors of the mesh nodes that surround the point of interest.

21. A method in accordance with claim 20,

further comprising repeating, at each of a plurality of mesh resolution levels, the acts of generating in the 2D x-ray image a mesh grid having a plurality of mesh nodes and estimating at least one nodal motion vector for each mesh node;

and

wherein the nodal motion vectors of the mesh nodes that surround the point of interest comprise nodal motion vectors that have been estimated at a final one of the plurality of mesh resolution levels.

22. A method in accordance with claim 21, further comprising:

determining whether any nodal motion vector that was estimated for any mesh node is unreliable, and if so, reconstructing the nodal motion vector for the mesh node.