Multimodality 2D To 3D Imaging Navigation

• US 20190325573A1
• Filed: 04/24/2018
• Published: 10/24/2019
• Est. Priority Date: 04/24/2018
• Status: Active Grant

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Bernard, Sylvain, Davis, Cynthia
• Granted Patent: US 10,796,430 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 6/025 Tomosynthesis
• CPC Class Codes: A61B 6/463 characterised by displaying multiple images or images and diagnostic data on one display
• CPC Class Codes: A61B 6/469 for selecting a region of interest [ROI]
• CPC Class Codes: A61B 6/481 involving the use of contrast agents
• CPC Class Codes: A61B 6/502 for diagnosis of breast, i.e. mammography
• CPC Class Codes: A61B 6/5223 generating planar views from image data, e.g. extracting a coronal view from a 3D image
• CPC Class Codes: A61B 6/5235 combining images from the same or different ionising radiation imaging techniques, e.g. PET and CT
• CPC Class Codes: A61B 6/5247 combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound
• CPC Class Codes: A61B 8/0825 for diagnosis of the breast, e.g. mammography
• CPC Class Codes: A61B 8/085 for locating body or organic structures, e.g. tumours, calculi, blood vessels, nodules
• CPC Class Codes: A61B 8/4218 characterised by articulated arms
• CPC Class Codes: A61B 8/4281 characterised by sound-transmitting media or devices for coupling the transducer to the tissue
• CPC Class Codes: A61B 8/5207 involving processing of raw data to produce diagnostic data, e.g. for generating an image reconstruction G06T11/00
• CPC Class Codes: A61B 8/54 Control of the diagnostic device
• CPC Class Codes: G06T 11/003 Reconstruction from projections, e.g. tomography
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/10084 Hybrid tomography; Concurrent acquisition with multiple different tomographic modalities
• CPC Class Codes: G06T 2207/10088 Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/10112 : Digital tomosynthesis [DTS]
• CPC Class Codes: G06T 2207/30068 : Mammography; Breast
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection

Abstract

A system and method for the detection of ROIs in images obtained of a breast or other tissue of a patient significantly improves the speed and precision/accuracy of navigation between multimodality 2D and 3D images. In the system and method, images of the tissue are obtained in a DBT acquisition to generate a synthetic 2D image of the imaged tissue and in a 3D, e.g., ultrasound, image acquisition. The 2D image generation process creates a synthetic 2D image that embed a navigation map correlating pixels in the 2D images to sections of the 3D ultrasound volume, such as via a registration between the 3D ultrasound volume and a 3D volume created using the DBT image data. When a synthetic 2D image is reviewed, an ROI on the 2D image is selected and the system will additionally present the user with the section of the 3D volume containing that ROI.

Claims

1. A method of navigating between images of an object obtained in different imaging modalities of the object, the method comprising:

obtaining a number of x-ray images of the object at varying angles relative to the object in a digital breast tomosynthesis (DBT) acquisition;

obtaining a three-dimensional (3D) volumetric image of the object in a 3D imaging acquisition;

forming a DBT volume of the object from the x-ray images;

forming a synthetic two-dimensional (2D) image of the object from the x-ray images and/or the DBT volume;

forming a navigation map correlating pixels of the synthetic 2D image with associated planes in the DBT volume;

correlating the 3D volumetric image from the 3D imaging acquisition with the DBT volume from the DBT acquisition;

displaying the synthetic 2D image; and

displaying a section of the 3D volumetric image corresponding to a location of a pixel in the synthetic 2D image in response to a selection of the pixel in the displayed synthetic 2D image.

2. A method according to claim 1 wherein the 3D imaging acquisition is an ultrasound (US) acquisition.

3. A method according to claim 1 wherein the 3D imaging acquisition is a magnetic resonance imaging (MRI) acquisition.

4. A method according to claim 1 wherein the 3D imaging acquisition is a contrast enhanced DBT (CEDBT) acquisition.

5. A method according to claim 1 wherein the 3D imaging acquisition is a CT acquisition.

6. A method according to claim 1 wherein the registration of the 3D volumetric image to the DBT volume comprises employing a finite element model.

7. A method according to claim 1 wherein the registration of the 3D volumetric image to the DBT volume comprises matching findings in the 3D volumetric image and the DBT volume.

8. A method according to claim 1 wherein the step of obtaining a number of x-ray images of the object and the step of obtaining a 3D volumetric image of the object occur with the object in the same position.

9. A method according to claim 1 wherein the step of forming the navigation map comprises associating a height value to each pixel of the synthetic 2D image corresponding to a position in the 3D volumetric image.

10. A method according to claim 1 wherein the step of forming the synthetic 2D image further comprises the steps of:

detecting one or more volumes of interest (VOIs) within the 3D volumetric image;

determining the positions of the one or more VOIs from the 3D volumetric image within the DBT volume;

reprojecting the VOIs from the DBT volume; and

modifying the navigation map to include the location of the VOIs.

11. A method according to claim 10 wherein the VOI reprojections are blended onto the synthetic 2D image after formation of the synthetic 2D image.

12. A method according to claim 10 wherein the VOI reprojections are utilized to enrich the synthetic 2D image during formation of the synthetic 2D image.

13. A method according to claim 10 further comprising the step of representing a boundary overlay of the VOI from the DBT volume within the synthetic 2D image.

14. A method according to claim 1 further comprising the steps of:

detecting a position of the nipple within the 3D volumetric image;

locating the position of the nipple from the 3D volumetric image into the DBT volume; and

representing the nipple in the synthetic 2D image.

15. A method according to claim 14 wherein the step of representing the nipple in the synthetic 2D image includes representing a distance from a selected pixel in the synthetic 2D image to the nipple.

16. A method according to claim 14 wherein the step of representing the nipple in the synthetic 2D image includes representing a clockwise position relative to the nipple of a selected pixel in the synthetic 2D image.

17. A method according to claim 1 further comprising the step of illustrating a cursor on the synthetic 2D image and the section of the 3D volumetric image on the display, wherein the cursor represents the location of the selected pixel in the synthetic 2D image and the corresponding position in the section of the 3D volumetric image.

18. A system for assisting a user in navigating through 2D and 3D image data to locate a region of interest within a breast tissue, the system comprising:

a 2D image acquisition system for acquiring 2D image data;

a 3D image acquisition system for acquiring 3D image data;

an analysis module operably connected to the 2D image acquisition system and the 3D image acquisition system and configured;

to generate a synthetic 2D image and a DBT volume from the acquired 2D image data;

to generate a 3D volume from the 3D image data;

to correlate locations within the DBT volume to locations within the 3D image volume;

to generate a navigation map providing information for each pixel in the synthetic 2D image corresponding to a plane in the DBT volume and the 3D image volume; and

to select and present a section within the 3D volume corresponding to the navigation map information for a pixel selected within the synthetic 2D image;

a display operably connected to the analysis module and operable to display the synthetic 2D image and sections of the 3D volume; and

an input operably connected to the analysis module and configured to receive commands from a user regarding the selection of a pixel within the synthetic 2D image present on the display.

19. A system according to claim 18 wherein the 3D image acquisition system is an automated breast ultrasound system (ABUS) or a magnetic resonance imaging (MRI) system.

20. A system according to claim 18 further comprising a computer aided detection system associated with the analysis module and configured with the analysis module to:

detect regions of interest within the 3D volume;

update the navigation map with the locations of the regions of interest (ROIs); and

optionally enhance the synthetic 2D image to illustrate the ROIs.

21. A system according to claim 18 wherein the analysis module presents a cursor on the synthetic 2D image and the section of the 3D volume on the display, wherein the cursor represents the location of the selected pixel in the synthetic 2D image and the section of the 3D volume.

22. A system according to claim 18 wherein the analysis module is further configured to:

detect a position of the nipple within the 3D volume;

locating the position of the nipple from the 3D volume into the DBT volume; and

represent the nipple and information on the location of the selected pixel relative to the nipple on the display in the synthetic 2D image formed from DBT volume and in the section of the 3D volume.