Seed localization system and method in ultrasound by fluoroscopy and ultrasound fusion
First Claim
1. A system for determining a position of at least one implanted object in a body, comprising:
- an ultrasound imager configured to forming an ultrasound image of a portion of the body containing the at least one implanted object;
a fluoroscopy imager configured to form a plurality of fluoroscopic images of the portion of the body; and
a computer system coupled to said ultrasound imager and to said fluoroscopy imager, said computer system processing the ultrasound image and the plurality of fluoroscopic images to calculate the position of the at least one implanted object in the body.
3 Assignments
0 Petitions
Accused Products
Abstract
A seed localization system and method in which a computer-based system is used to determine the three-dimensional (3D) position of radiotherapy seeds with respect to an area of affected tissue, such as the prostate, using ultrasound (US) and fluoroscopy (FL) imaging, so that a radiotherapy dose may be calculated. One embodiment the present invention may be used to determine the 3D position of implanted brachytherapy seeds. An alternative embodiment of the invention may be used to determine the 3D position of implanted objects other than brachytherapy seeds. The seed localization system and method includes a graphical user interface useful for assisting a user of the seed localization system in its operation.
-
Citations
60 Claims
-
1. A system for determining a position of at least one implanted object in a body, comprising:
-
an ultrasound imager configured to forming an ultrasound image of a portion of the body containing the at least one implanted object;
a fluoroscopy imager configured to form a plurality of fluoroscopic images of the portion of the body; and
a computer system coupled to said ultrasound imager and to said fluoroscopy imager, said computer system processing the ultrasound image and the plurality of fluoroscopic images to calculate the position of the at least one implanted object in the body. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 14)
a processor for processing the ultrasound image and the plurality of fluoroscopic images; and
a monitor coupled to said processor and configured to display a three-dimensional image of the portion of the body showing the position of the at least one implanted object in the body.
-
-
3. The system of claim 2, wherein said computer system further includes a graphical user interface coupled to said processor, said graphical user interface enabling a user to interact with said processor.
-
4. The system of claim 3, wherein said graphical user interface includes:
-
a first data input adapted to receive data regarding the ultrasound image;
a second data input adapted to receive data regarding the plurality of fluoroscopic images; and
a data analyzer coupled to said first data input and to said second data input and adapted to calculate from the ultrasound image a series of three-dimensional coordinates Q1, Q2, . . . , QM associated with M markers placed in the portion of the body and visible in the ultrasound image and the plurality of fluoroscopic images, wherein M≧
4.
-
-
5. The system of claim 4, wherein said data analyzer is further adapted to calculate:
-
at least one set of two-dimensional coordinates for the at least one implanted object in each of the plurality of fluoroscopic images; and
M sets of two-dimensional coordinates for the M markers in each of the plurality of fluoroscopic images.
-
-
6. The system of claim 5, wherein said graphical user interface further includes a coordinate reconstructor coupled to said data analyzer and adapted to determine:
-
a series of three-dimensional coordinates R1, R2, . . . , RN associated with N implanted objects; and
a series of three-dimensional coordinates P1, P2, . . . , PM associated with the M markers.
-
-
7. The system of claim 6, wherein said graphical user interface further includes a coordinate correlator adapted to associate each of the series of three-dimensional coordinates Pi with each of the series of three-dimensional coordinates Qi for each of the M makers, wherein 1≦
- i≦
M.
- i≦
-
8. The system of claim 7, wherein said coordinate correlator is further adapted to determine a 3×
- 3 matrix T and a 3×
1 vector t by solving an optimization problem.
- 3 matrix T and a 3×
-
9. The system of claim 8, wherein:
-
an initial estimate for (T,t) is found by solving a first optimization problem a subsequent estimate for (T,t) is found by solving a second optimization problem
wherein I(X) is a scalar intensity of point X in the ultrasound image; and
if the second optimization problem has no unique solution, the subsequent estimate for (T,t) is found through a locally optimal solution.
-
-
10. The system of claim 8, wherein said coordinate correlator is further adapted to map each of the series of three-dimensional coordinates R1, R2, . . . , RN to a series of three-dimensional coordinates S1,S2, . . , SN by a transformation Sj=TRj+t, wherein 1≦
- j≦
N.
- j≦
-
11. The system of claim 1, wherein the at least one implanted object includes a plurality of brachytherapy seeds used in a radiation treatment of affected tissue.
-
14. The system of claim 6 wherein said graphical user interface further comprises:
a coordinate reconstructor adapted to determine a series R1, R2, . . . , RN and P1, P2, . . . , PM where Ri and Pi correspond to a unique set of derived three dimensional coordinates associated with each implanted seed and marker, respectively, appearing in said plurality of two dimensional fluoroscopic images.
-
12. A system for determining the three dimensional position of implanted objects, comprising:
-
a computer system adapted to receive a three dimensional ultrasound image of a region containing the implanted objects and a plurality of two dimensional fluoroscopic images of the region, said computer system being adapted to form from said three dimensional ultrasound image and said plurality of two dimensional fluoroscopic images an improved three dimensional image of the region, said improved three dimensional image capable of indicating the location of each of the implanted objects; and
a graphical user interface for determining the three dimensional position of the implanted objects with respect to the region, wherein said graphical user interface prompts and coordinates execution of a sequence of steps performed cooperatively by a user and said computer system and further comprises;
a data input adapted to receive a number M corresponding to a number of implanted markers and a number N corresponding to a number of the implanted objects; and
a data analyzer adapted to;
locate the M highly visible implanted markers within the three dimensional ultrasound image, where M≧
4; and
store on a computer-readable medium a series Q1, Q2, . . . , QM for 1≧
i≧
M wherein Qi corresponds to a unique set of three dimensional coordinates associated with each of the M highly visible markers.- View Dependent Claims (13, 15, 16, 17, 18)
locate each implanted seed and marker appearing in each image of said plurality of two dimensional fluoroscopic images; and
store on a computer-readable medium a unique set of two dimensional coordinates corresponding to the location of each implanted seed and marker appearing in each of said two dimensional fluoroscopic images.
-
-
15. The system of claim 13 wherein said graphical user interface further comprises:
a coordinate correlator adapted to associate each said unique set of three dimensional FL coordinates P1 for 1≦
i≦
M corresponding to the location of each of the M highly visible markers with each said unique set of identified three dimensional US coordinates Qi for 1≦
i≦
M corresponding to the same marker.
-
16. The system of claim 15 wherein said coordinate correlator is further adapted to:
map each said unique set of derived three dimensional FL coordinates Ri. corresponding to an implanted seed to the three dimensional US coordinates Sicorresponding to the same implanted seed by the transformation Si=TRi+t.
-
17. The system of claim 16 wherein said coordinate correlator is further adapted to determine a solution to an optimization problem.
-
18. The system of claim 16 wherein said coordinate correlator is further adapted to:
-
determine a solution 3×
3 matrix Tand a 3×
1 vector t wherein;
an initial estimate for (T,t) is found by determining the unique solution to the optimization problem a final estimate is found by solving the optimization problem and if the maximization problem has no unique solution, a locally optimal solution is determined.
-
-
19. A method for locating a plurality of implanted seeds, comprising the steps of:
-
obtaining a three-dimensional ultrasound image of a region containing the plurality of implanted seeds;
obtaining a plurality of two-dimensional fluoroscopic images of the region;
matching the plurality of implanted seeds in the three-dimensional ultrasound image with corresponding ones in the plurality of two-dimensional fluoroscopic images; and
calculating a plurality of three-dimensional coordinates of the plurality of implanted seeds by analyzing the three-dimensional ultrasound image and the plurality of two-dimensional fluoroscopic images. - View Dependent Claims (20, 21, 22, 23, 24, 25, 26)
locating the M markers within the three-dimensional ultrasound image, wherein M≧
4; and
calculating a series of three-dimensional coordinates Q1, Q2, . . . , QM of the M markers by analyzing the three-dimensional ultrasound image.
-
-
23. The method of claim 22, further comprising the steps of:
-
locating the plurality of implanted seeds and the M markers in each of the plurality of two-dimensional fluoroscopic images;
calculating a first plural sets of two-dimensional coordinates for the plurality of implanted seeds appearing in the plurality of two-dimensional fluoroscopic images;
calculating a second plural sets of two-dimensional coordinates for the M markers appearing in the plurality of two-dimensional fluoroscopic images;
determining a first series of three-dimensional coordinates R1, R2, . . . , RN of the plurality of implanted seeds from the first plural sets of two-dimensional coordinates, wherein N is a number of the plurality of implanted seeds; and
determining a second series of three-dimensional coordinates P1, P2, . . . , PM of the M markers from the second plural sets of two-dimensional coordinates.
-
-
24. The method of claim 23, further comprising the step of associating each Pi with a corresponding Qi, wherein 1≦
- i≦
M.
- i≦
-
25. The method of claim 24, further comprising the step of finding a series of three-dimensional coordinates S1, S2, . . . , SN of the N implanted seeds by a transformation Sj=TRj+t, wherein 1≦
- j≦
N.
- j≦
-
26. The method of claim 25, wherein the step of associating includes finding a 3×
- 3 matrix T and a 3×
1 vector t by determining a solution to an optimization problem.
- 3 matrix T and a 3×
-
27. A method for determining the three dimensional position of implanted seeds, comprising the steps of:
-
inputting a number M corresponding to a number of implanted markers and a number N corresponding to a number of implanted seeds;
obtaining a three dimensional ultrasound image of a region of implanted seeds;
obtaining a plurality of two dimensional fluoroscopic images of the region of implanted seeds;
forming an improved three dimensional image of the region of implanted seeds by analyzing data from said three dimensional ultrasound image in combination with data from said plurality of two dimensional fluoroscopic images; and
identifying the location of each implanted seed in the region by analysis of said improved three dimensional image.
-
-
28. A method for determining the three dimensional position of implanted seeds, comprising the steps of:
-
obtaining a three dimensional ultrasound image of a region of implanted seeds;
obtaining a plurality of two dimensional fluoroscopic images of the region of implanted seeds;
locating M highly visible implanted markers within the three dimensional ultrasound image, where M≧
4;
storing on a computer-readable medium a series Q1, Q2, . . . , QM for 1≧
i≧
M wherein Qi corresponds to a unique set of three dimensional coordinates associated with each of the M highly visible markers;
forming an improved three dimensional image of the region of implanted seeds by analyzing data from said three dimensional ultrasound image in combination with data from said plurality of two dimensional fluoroscopic images; and
identifying the location of each implanted seed in the region by analysis of said improved three dimensional image.
-
-
29. A method for determining the three dimensional position of implanted seeds, comprising the steps of:
-
obtaining a three dimensional ultrasound image of a region of implanted seeds;
obtaining a plurality of two dimensional fluoroscopic images of the region of implanted seeds;
locating each implanted seed and marker appearing in each image of said plurality of two dimensional fluoroscopic images;
storing on a computer-readable medium a unique set of two dimensional coordinates corresponding to the location of each implanted seed and marker appearing in each said two dimensional fluoroscopic image;
forming an improved three dimensional image of the region of implanted seeds by analyzing data from said three dimensional ultrasound image in combination with data from said plurality of two dimensional fluoroscopic images;
identifying the location of each implanted seed in the region by analysis of said improved three dimensional image; and
determining series R1, R2, . . . , R and P1, P2, . . . , PM where Ri and Pi correspond to a unique set of derived three dimensional FL coordinates associated with each implanted seed and marker, respectively, appearing in said plurality of two dimensional fluoroscopic images. - View Dependent Claims (30)
associating each said unique set of three dimensional FL coordinates Pi for 1≦
i≦
M corresponding to the location of each of the M highly visible markers with each said unique set of identified three dimensional US coordinates Qi for 1≦
i≦
M corresponding to the same implanted marker.
-
-
31. A method for determining the three dimensional position of implanted seeds, comprising the steps of:
-
obtaining a three dimensional ultrasound image of a region of implanted seeds;
obtaining a plurality of two dimensional fluoroscopic images of the region of implanted seeds;
forming an improved three dimensional image of the region of implanted seeds by analyzing data from said three dimensional ultrasound image in combination with data from said plurality of two dimensional fluoroscopic images, wherein said step of forming an improved three dimensional image further comprises mapping each said unique set of derived three dimensional FL coordinates Ri, corresponding to an implanted seed to its 3D US location Si; and
identifying the location of each implanted seed in the region by analysis of said improved three dimensional image. - View Dependent Claims (32, 33)
determining a solution 3×
3 matrix T and a 3×
1 vector t wherein;
an initial estimate for (T,t) is found by determining the unique solution to the optimization problem a final estimate is found by solving the optimization problem if the maximization problem has no unique solution, a locally optimal solution is determined.
-
-
34. A computer-generated graphical user interface for determining positions of a plurality of brachytherapy seeds with respect to an implanted region, the graphical user interface prompting and coordinating execution of a sequence of steps performed cooperatively by a user and a computer processor, said sequence of steps comprising:
-
obtaining a three-dimensional ultrasound image of the implanted region;
obtaining a plurality of two-dimensional fluoroscopic images of the implanted region;
forming an improved three-dimensional image of the implanted region by analyzing the three-dimensional ultrasound image in combination with the plurality of two-dimensional fluoroscopic images; and
identifying a position for each of the plurality of brachytherapy seeds in the implanted region in the improved three-dimensional image. - View Dependent Claims (35, 36, 37, 38, 39, 40, 41, 42)
locating the M markers within the three-dimensional ultrasound image, where M≧
4; and
storing on a computer-readable medium a series Q1, Q2, . . . , QM of three-dimensional coordinates associated with the M markers.
-
-
38. The graphical user interface of claim 37, wherein said sequence of steps further comprises:
-
locating the N brachytherapy seeds and the M markers appearing in each of said plurality of two-dimensional fluoroscopic images;
storing on the computer-readable medium a plural sets of two-dimensional coordinates for the N brachytherapy seeds and the M markers appearing in each of said plurality of two-dimensional fluoroscopic images; and
deriving a first series R1, R2, . . . , RN of three-dimensional coordinates for the N brachytherapy seeds and a second series P1, P2, . . . , PM of three-dimensional coordinates for the M markers from the plural sets of two-dimensional coordinates.
-
-
39. The graphical user interface of claim 38, wherein said sequence of steps further comprises associating each Pi with a corresponding Qi for 1≦
- i≦
M .
- i≦
-
40. The graphical user interface of claim 39, wherein said step of associating further comprises determining a solution to an optimization problem.
-
41. The graphical user interface of claim 38, wherein said sequence of steps further comprises finding a matrix T and a 3×
- 1 vector t through an optimization process, wherein;
an initial estimate for (T,t) is found by solving a minimization problem a second estimate for (T,t) is found by solving a maximization
wherein I(X) is a scalar intensity of point X in the three-dimensional ultrasound image; and
if the maximization problem has no unique solution, the second estimation for (T,t) is found through a locally optimal solution.
- 1 vector t through an optimization process, wherein;
-
42. The graphical user interface of claim 41, wherein said sequence of steps further comprises calculating a series of derived three-dimensional coordinates S1, S2, . . . , SN of the N brachytherapy seeds, wherein Sj=TRj+t for 1≦
- j≦
N.
- j≦
-
43. A computer-generated graphical user interface for determining the three dimensional position of brachytherapy seeds with respect to an implanted region wherein said graphical user interface prompts and coordinates execution of a sequence of steps performed cooperatively by a user and a computer processor, said sequence of steps comprising:
-
inputting a number M corresponding to a number of implanted markers and a number N corresponding to the number of implanted seeds;
obtaining a three dimensional ultrasound image of a region of implanted seeds;
obtaining a plurality of two dimensional fluoroscopic images of the region of implanted seeds;
forming an improved three dimensional image of the region of implanted seeds by analyzing data from said three dimensional ultrasound image in combination with data from said plurality of two dimensional fluoroscopic images; and
identifying the location of each implanted seed in the region by analysis of said improved three dimensional image.
-
-
44. A computer-generated graphical user interface for determining the three dimensional position of brachytherapy seeds with respect to an implanted region wherein said graphical user interface prompts and coordinates execution of a sequence of steps performed cooperatively by a user and a computer processor, said sequence of steps comprising:
-
obtaining a three dimensional ultrasound image of a region of implanted seeds;
obtaining a plurality of two dimensional fluoroscopic images of the region of implanted seeds;
locating M highly visible implanted markers within the three dimensional ultrasound image, where M≧
4;
storing on a computer-readable medium a series Q1, Q2, . . . , QMfor 1≦
i≦
M wherein Qi corresponds to a unique set of three dimensional coordinates associated with each of the at least four highly visible markers;
forming an improved three dimensional image of the region of implanted seeds by analyzing data from said three dimensional ultrasound image in combination with data from said plurality of two dimensional fluoroscopic images; and
identifying the location of each implanted seed in the region by analysis of said improved three dimensional image.
-
-
45. A computer-generated graphical user interface for determining the three dimensional position of brachytherapy seeds with respect to an implanted region wherein said graphical user interface prompts and coordinates execution of a sequence of steps performed cooperatively by a user and a computer processor, said sequence of steps comprising:
- obtaining a three dimensional ultrasound image of a region of implanted seeds;
obtaining a plurality of two dimensional fluoroscopic images of the region of implanted seeds;
locating each implanted seed and marker appearing in each image of said plurality of two dimensional fluoroscopic images;
storing on a computer-readable medium a unique set of two dimensional coordinates corresponding to the location of each implanted seed and marker appearing in each said two dimensional fluoroscopic image;
determining a series R1, R2, . . . , RN and P1, P2, . . . , PM where Ri and Pi correspond to a unique set of derived three dimensional coordinates associated with each implanted seed and marker, respectively, appearing in said plurality of two dimensional fluoroscopic images;
forming an improved three dimensional image of the region of implanted seeds by analyzing data from said three dimensional ultrasound image in combination with data from said plurality of two dimensional fluoroscopic images; and
identifying the location of each implanted seed in the region by analysis of said improved three dimensional image. - View Dependent Claims (46, 47, 48, 49)
associating each said unique set of derived three dimensional FL coordinates P, for 1≦
i≦
M corresponding to the location of each of the M highly visible markers with each said unique set of identified three dimensional coordinates Qi for 1≦
i≦
M corresponding to the same marker.
- obtaining a three dimensional ultrasound image of a region of implanted seeds;
-
47. The graphical user interface of claim 45 wherein said graphical user interface further prompts and coordinates the step of:
mapping each said unique set of derived three dimensional FL coordinates Ri corresponding to an implanted seed to its 3D ultrasound coordinate Si.
-
48. The graphical user interface of claim 47 wherein said step of associating further comprises determining a solution to an optimization problem.
-
49. The graphical user interface of claim 48 wherein said step of determining a solution to an optimization problem further comprises:
-
determining a solution 3×
3 matrix Tand a 3×
1 vector t wherein;
an initial estimate for (T,t) is found by determining the unique solution to the optimization problem a final estimate is found by solving the optimization problem
andif the maximization problem has no unique solution, a locally optimal solution is determined.
-
-
50. A computer-readable medium on which is embodied a set of programmed instructions that causes a processor to perform a sequence of steps, said sequence of steps comprising:
-
obtaining a three-dimensional ultrasound image of a region containing a plurality of implanted seeds;
obtaining a plurality of two-dimensional fluoroscopic images of the region;
forming an improved three-dimensional image of the region by analyzing the three-dimensional ultrasound image in combination with the plurality of two-dimensional fluoroscopic images; and
identifying a location for each of the plurality of implanted seeds in the region by analysis of the improved three-dimensional image.
-
-
51. A process for locating a plurality of objects in a region, comprising the steps of:
-
placing a plurality of markers in the region, the plurality of markers being visible via a first imaging mode and a second imaging mode and being distinguishable from the plurality of objects;
forming a first image of the first imaging mode of the region;
identifying a first plurality of markings in the first image corresponding to the plurality of markers;
forming a second image and a third image of the second imaging mode of the region;
identifying a second plurality of markings in the second image corresponding to the plurality of markers;
identifying a third plurality of markings in the third image corresponding to the plurality of markers;
establishing a correlation between the first image, the second image, and the third image by matching the first plurality of markings with the second plurality of markings and the third plurality of markings;
deriving a set of object coordinates corresponding to the plurality of objects in the second image and in the third image; and
calculating a series of three-dimensional coordinates for the plurality of objects in response to the correlation between the first image, the second image, and the third image and to the set of object coordinates. - View Dependent Claims (52, 53, 54, 55, 56, 57, 58, 59, 60)
said step of forming a first image of the first imaging mode includes forming a three-dimensional ultrasound image of the region; and
said step of forming a second image and a third image of the second imaging mode includes forming a plurality of two-dimensional fluoroscopic images of the region.
-
-
55. The process of claim 51, wherein:
-
said step of identifying a first plurality of markings includes deriving a first set of marker coordinates Q1, Q2, . . . , QM for the plurality of markers, wherein M≧
4 is a number of the markers;
said steps of identifying a second plurality of markings identifying a third plurality of markings include deriving a second set of marker coordinates P1, P2, . . . , PM for the M markers; and
said step of deriving a set of coordinates corresponding to the plurality of objects in the second image and in the third image includes deriving a set of object coordinates R1, R2, . . . , RN, wherein N is a number of the plurality of objects.
-
-
56. The process of claim 55, wherein said step establishing a correlation between the first image, the second image, and the third image includes a step of finding a 3×
- 3 matrix Tand a 3×
1 vector t by solving a first optimization problem
- 3 matrix Tand a 3×
-
57. The process of claim 56, wherein said step of finding a 3×
- 3 matrix Tand a 3×
1 vector t further includes solving a second optimization problemwherein I(X) is a scalar intensity at a point X in the first image.
- 3 matrix Tand a 3×
-
58. The process of claim 57, wherein said step of finding a 3×
- 3 matrix Tand a 3×
1 vector t further includes solving a localized optimization problem in response to the second optimization problem not having a unique solution.
- 3 matrix Tand a 3×
-
59. The process of claim 56, wherein said step of calculating a series of three-dimensional coordinates for the plurality of objects includes deriving the series of three-dimensional coordinates S1, S2, . . . , SN of the N implanted seeds by a transformation Sj=TRj+t, wherein 1≦
- j≦
N .
- j≦
-
60. The process of claim 51, generating a three-dimensional visual display of the region indicating the plurality of objects in accordance with the series of three-dimensional coordinates for the plurality of objects.
Specification