SYSTEM AND METHOD OF OVERLAYING IMAGES OF DIFFERENT MODALITIES

• US 20140044325A1
• Filed: 08/09/2012
• Published: 02/13/2014
• Est. Priority Date: 08/09/2012
• Status: Active Grant

• Current Assignee: Hologic Incorporated
• Original Assignee: Hologic Incorporated
• Inventors: Ma, Kayan, Gill, Sean
• Granted Patent: US 9,076,246 B2
• Time in Patent Office: Days
• US Class Current: 382/128
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10132 Ultrasound image
• CPC Class Codes: G06T 2207/10136 3D ultrasound image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

Systems and methods of overlaying a three-dimensional image of a tissue and an image of the tissue from an imaging transducer, having a field of view are disclosed. The method includes determining pixel dimensions of the field of view in the second co-ordinate space, scaling the three-dimensional image in the first co-ordinate space to match the pixel dimensions of the field of view in the second co-ordinate space, and displaying an overlaid image comprising the three-dimensional image and the field of view of the imaging transducer.

Claims

1. A method overlaying a three-dimensional image of a tissue and an image of the tissue from an imaging transducer, having a field of view, the method comprising:

co-registering a first co-ordinate space of the three-dimensional image with a second co-ordinate space of the field of view of the image from the imaging transducer;

determining pixel dimensions of the field of view in the second co-ordinate space;

scaling the three-dimensional image in the first co-ordinate space to match the pixel dimensions of the field of view in the second co-ordinate space;

transforming position and orientation of the three-dimensional image of the tissue in the first co-ordinate space to match position and orientation of the field of view in the second co-ordinate space; and

displaying an overlaid image comprising the three-dimensional image and the field of view of the imaging transducer.

2. The method of claim 1, wherein determining pixel dimensions of the field of view further comprises:

determining pixel coordinates of points of interest on an upper boundary of the field of view; and

determining pixel width and height based on geometry of the upper boundary of the field of view.

3. The method of claim 2, wherein the imaging transducer comprises a linear transducer, the upper boundary comprises a line segment, and the points of interest comprise ends points and a midpoint of the line segment.

4. The method of claim 3, wherein determining the pixel width and height based on geometry further comprises determining the pixel width and height based on geometry of the line segment and aspect ratio of the image.

5. The method of claim 2, wherein the imaging transducer comprises a curvilinear transducer, the upper boundary comprises an arc segment, and the points of interest comprise a midpoint and a radius of curvature.

6. The method of claim 5, wherein determining the pixel width and height based on geometry further comprises determining the pixel width and height based on geometry of the arc segment and aspect ratio of the image.

7. The method of claim 1, further comprising detecting changes in image depth of the field of view of the image and rescaling the field of view based on a scaling factor to match an adjusted image depth of the field of view.

8. The method of claim 1, further comprising:

receiving the three-dimensional image of the tissue from a first imaging system; and

receiving the image of the tissue from the imaging transducer from a second imaging system, wherein the first imaging system is different from the second imaging system.

9. An apparatus for overlaying a three-dimensional image of a tissue and an image of the tissue from an imaging transducer, having a field of view, the apparatus comprising:

a non-transitory computer readable medium configured to store any of the three-dimensional image and the image from the imaging transducer;

a processor, coupled to the computer readable medium and configured to;

co-register a first co-ordinate space of the three-dimensional image with a second co-ordinate space of the field of view of the image from the imaging probe;

determine pixel dimensions of the field of view in the second co-ordinate space;

scale the three-dimensional image in the first co-ordinate space to match the pixel dimensions of the field of view in the second co-ordinate space; and

transform position and orientation of the three-dimensional image in the first co-ordinate space to match the position and orientation of the field of view in the second co-ordinate space; and

a display configured to display an overlaid image comprising the three-dimensional image and the field of view of the imaging transducer.

10. The apparatus of claim 9, wherein the processor is configured to:

determine pixel coordinates of points of interest on an upper boundary of the field of view; and

determine pixel width and height based on geometry of the of the upper boundary of the field of view.

11. The apparatus of claim 10, wherein the imaging transducer comprises a linear transducer, the upper boundary comprises a line segment, and the points of interest comprise ends points and a midpoint of the line segment.

12. The apparatus of claim 10, wherein the imaging transducer comprises a curvilinear transducer, the upper boundary comprises an arc segment, and the points of interest comprise a midpoint and a radius of curvature.

13. The apparatus of claim 9, wherein the processor is further configured to detect changes in image depth of the field of view of the image and rescale the field of view based on a scaling factor to match an adjusted image depth of the field of view.

14. The apparatus of claim 9, further comprising an input module configured to receive the three-dimensional image of the tissue from a first imaging system and configured to receive the image of the tissue from the imaging transducer from a second imaging system, wherein the first imaging system is different from the second imaging system.

15. A method overlaying an image of a tissue from an imaging system, having an imaging transducer with a field of view, and a three-dimensional image of the tissue, the method comprising:

co-registering a first co-ordinate space of the three-dimensional image with a second co-ordinate space of the field of view of the image from the imaging transducer;

receiving input bounds of the field of view and depth markers identified in the image;

determining pixel spacing based on depth associated with the depth markers in the image; and

scaling the three-dimensional image in the first co-ordinate space to match the pixel spacing of the field of view in the second co-ordinate space;

transforming position and orientation of the three-dimensional image of the tissue in the first co-ordinate space to match the position and orientation of the field of view in the second co-ordinate space; and

displaying an overlaid image comprising the field of view of the imaging transducer and the three-dimensional image.

16. The method of claim 15, further comprising receiving depth associated with each of the depth markers in the field of view.

17. The method of claim 15, further comprising receiving input identifying a freeze state of the image from the transducer.

18. The method of claim 15, further comprising determining co-ordinates of points of interest defining center, upper and lower bounds of the field of view and determining pixel spacing based on the co-ordinates of the point of interest.

19. The method of claim 15, further comprising determining whether the field of view of the image is cropped.

20. The method of claim 15, wherein the imaging transducer comprises a curvilinear transducer, and the method further comprises detecting an angle of the field of view of the curvilinear transducer.

21. The method of claim 15, further comprising determining a set of pixels configured to define the field of view and masking out another set of pixels outside the defined the set of pixels defining the field of view.

22. The method of claim 15, further comprising:

receiving the three-dimensional image of the tissue from a first imaging system; and

receiving the image of the tissue from the imaging transducer from a second imaging system, wherein the first imaging system is different from the second imaging system.

23. An apparatus for overlaying an image of a tissue from an imaging system, having an imaging transducer with a field of view, and a three-dimensional image of the tissue, the apparatus comprising:

a non-transitory computer readable medium configured to store any of the three-dimensional image and the image from the imaging transducer;

a processor coupled to the computer readable medium and configured to;

co-register a first co-ordinate space of the three-dimensional image with a second co-ordinate space of the field of view of the image from the imaging transducer;

receive input identifying bounds of the field of view and depth markers in the image;

determine pixel spacing based on depth associated with the depth markers in the image; and

scale the three-dimensional image in the first co-ordinate space to match the pixel spacing of the field of view in the second co-ordinate space; and

transform position and orientation of the three-dimensional image in the first co-ordinate space to match the position and orientation of the field of view in the second co-ordinate space; and

a display configured to display an overlaid image comprising the field of view of the imaging transducer and the three-dimensional image.

24. The apparatus of claim 23, further comprising an input module configured to receive the three-dimensional image of the tissue from a first imaging system and configured to receive the image of the tissue from the imaging transducer from a second imaging system, wherein the first imaging system is different from the second imaging system.

25. A method of calibrating an imaging system for overlaying an image of a tissue from the imaging system and a three-dimensional image of the tissue, the imaging system having an imaging transducer with a field of view, the method comprising:

providing for a user to select bounds of the field of view of the imaging transducer;

providing for the user to select depth marker indicators in the image;

cycling through the depth marker indicators to determine a specific depth marker for each depth; and

calculating coordinates of points of interest in the field of view of the image; and

calculating pixel spacing based on the coordinates of the points of interest and depth associated with the depth markers in the image.

26. The method of claim 25, further comprising providing for the user to select an area identifying a freeze state of the image from the transducer.

27. The method of claim 25, further comprising providing for the user to select a background threshold, wherein the background threshold is used to filter an ultrasound signal overlapping with the depth marker indicators.

28. The method of claim 25, further comprising determining whether the field of view of the image is cropped.

29. The method of claim 25, wherein the imaging transducer comprises a curvilinear transducer, and the method further comprises detecting an angle of the field of view of the curvilinear transducer.