Breast cancer screening with ultrasound image overlays

• US 7,103,205 B2
• Filed: 11/27/2002
• Issued: 09/05/2006
• Est. Priority Date: 11/24/2000
• Status: Active Grant

• Current Assignee: U-System Inc. (General Electric Company)
• Original Assignee: U-System Inc. (General Electric Company)
• Inventors: Wang, Shih-Ping, Rao, Fangyi
• Primary Examiner(s): Johns, Andrew W.
• Application Number:
  US10/305,661
• Publication Number:
  US 20040068170A1
• Time in Patent Office: 1,378 Days
• Field of Search: 382/128, 382/132, 382/284, 382/294, 378/37, 378/63, 600/437, 600/440, 600/443
• US Class Current: 382/132
• CPC Class Codes: A61B 6/463 characterised by displaying multiple images or images and diagnostic data on one display
• CPC Class Codes: A61B 6/502 for diagnosis of breast, i.e. mammography
• CPC Class Codes: A61B 6/5247 combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound
• CPC Class Codes: A61B 6/563 involving image data transmission via a network
• CPC Class Codes: A61B 8/0825 for diagnosis of the breast, e.g. mammography
• CPC Class Codes: A61B 8/406 using means for diagnosing suspended breasts
• CPC Class Codes: A61B 8/4281 characterised by sound-transmitting media or devices for coupling the transducer to the tissue
• CPC Class Codes: A61B 8/4405 Device being mounted on a trolley
• CPC Class Codes: A61B 8/462 characterised by constructional features of the display
• CPC Class Codes: A61B 8/463 characterised by displaying multiple images or images and diagnostic data on one display composite display in devices for short-range imaging with acoustic waves G01S7/52074
• CPC Class Codes: A61B 8/464 involving a plurality of displays
• CPC Class Codes: A61B 8/465 adapted to display user selection data, e.g. icons or menus
• CPC Class Codes: A61B 8/467 characterised by special input means
• CPC Class Codes: A61B 8/5238 for combining image data of patient, e.g. merging several images from different acquisition modes into one image for image registration algorithms G06T7/30
• CPC Class Codes: A61B 8/565 involving data transmission via a network
• CPC Class Codes: Y10S 128/916 Ultrasound 3-D imaging

Abstract

An adjunctive ultrasound mammography system and associated methods are described in which an ultrasound image being displayed near an x-ray mammogram image may be superimposed thereon or thereunder by a user for facilitating rapid comprehension of breast structures and detection of abnormalities therein. In one preferred embodiment, the x-ray mammogram image corresponds to a standard x-ray mammogram view, and the ultrasound image is a thick-slice image representing a thick-slice or slab-like portion of the breast volume substantially parallel to that standard x-ray mammogram view. In another preferred embodiment, the user is permitted to manually manipulate the registration of the ultrasound image with the x-ray mammogram image. It has been found that the manual registration process, which involves manual vernier adjustments responsive to perceived registration differences, can rapidly increase the radiologist'"'"'s perception of the breast structures being displayed by both component images. Even though ultrasound images tend to have substantially different textures and feature emphases than x-ray images, the bimodal thick-slice/x-ray image, alone or in conjunction with the vernier registration process, can often expose or clarify tissue structures hidden in the separate component images, and can often obviate or explain certain noticed structures in the component images.

Claims

1. A method for facilitating the detection of abnormalities in a breast, comprising:

displaying an x-ray mammogram image of the breast to a user;

displaying a thick-slice ultrasound image of the breast near said x-ray mammogram image, said thick-slice ultrasound image representing sonographic properties of a slab-like volume within the breast occupying less than the entire breast volume; and

overlaying one of said ultrasound image or said x-ray mammogram image over the other to form a bimodal image, said bimodal image facilitating user perception of structures within the breast.

2. The method of claim 1, said x-ray mammogram image corresponding to a standard x-ray mammogram view plane, said slab-like volume being substantially parallel to said standard x-ray mammogram view plane.

3. The method of claim 2, said thick-slice ultrasound image being displayed on an electronic display device, further comprising dynamically varying a thickness of said slab-like volume responsive to a first user control input.

4. The method of claim 2, further comprising dynamically varying an elevation of said slab-like volume relative to a reference plane parallel to said standard x-ray mammogram view plane responsive to a second user control input.

5. The method of claim 1, said one of said ultrasound image or said x-ray mammogram image being an overlying image and said other being an underlying image, said overlaying further comprising:

moving said overlying image onto said underlying image from an initial nearby position such that said overlying and underlying images are in at least approximate registration; and

performing vernier shifts to a position of said overlying image relative to said underlying image under manual control of the user, said vernier shifts facilitating user perception of structures within the breast.

6. The method of claim 5, one of said overlying or underlying images being displayed on an electronic display device, wherein said moving said overlying image is executed by a processor according to an automated registration method.

7. The method of claim 5, wherein said moving said overlying image is executed under manual control of the user.

8. The method of claim 7, said moving said overlying image comprising:

identifying at least one natural watermark within the breast tissue visible on both of said overlying and underlying images; and

aligning said overlying image over said underlying image using said natural watermark.

9. The method of claim 8, said natural watermark comprising a calcification.

10. The method of claim 5, said performing vernier shifts comprising, if said overlying and underlying images are already in perfected registration, shifting said overlying image by a small amount such that said images are not in perfected registration, and then shifting said overlying image back into perfected registration, said shifting facilitating user perception of structures within the breast.

11. The method of claim 5, said overlying and underlying images both being hardcopy images, the overlying image being imprinted upon a translucent medium to allow at least partial visibility of the underlying image, said overlaying being performed by direct hand control of the hardcopy images.

12. The method of claim 5, said underlying image being displayed on a computer monitor, said overlying image being displayed in hardcopy format on a translucent medium.

13. The method of claim 5, said overlying and underlying images both being displayed on a computer monitor, said overlaying comprising pixelwise mixing of said overlying and underlying images according to a predetermined mixing algorithm.

14. The method of claim 13, wherein said manual control is implemented using a control device selected from the group consisting of:

joystick, trackball, computer mouse, and touchscreen.

15. The method of claim 13, wherein said predetermined mixing algorithm comprises at least one mixing factor that is dynamically user-adjustable, user adjustment of said mixing factor also facilitating user perception of the breast structures.

16. A method for facilitating the detection of abnormalities in a breast, comprising:

displaying an x-ray mammogram image of the breast to a user;

displaying an ultrasound image of the breast near said x-ray mammogram image;

overlaying one of said ultrasound image or said x-ray mammogram image over the other such that said images are in at least approximate registration; and

shifting a position of said overlying image relative to said underlying image by incremental amounts under manual control of the user, said shifting facilitating user perception of structures within the breast.

17. The method of claim 16, said one of said ultrasound image or said x-ray mammogram image being an overlying image and said other being an underlying image, said overlaying comprising moving said overlying image into said approximate registration with said underlying image, said shifting by incremental amounts being performed subsequent to said moving into approximate registration.

18. The method of claim 17, wherein said moving into approximate registration is executed by a processor according to an automated registration method.

19. The method of claim 17, said shifting by incremental amounts comprising, if said overlying and underlying images are in substantially perfect registration, perturbing and then restoring said substantially perfect registration, said perturbing and restoring facilitating user perception of structures within the breast.

20. The method of claim 17, wherein said moving into approximate registration is executed under manual control of the user.

21. The method of claim 20, said moving into approximate registration comprising:

identifying at least one natural watermark within the breast tissue visible on both of said overlying and underlying images; and

aligning said overlying image over said underlying image using said natural watermark.

22. The method of claim 21, said natural watermark comprising a calcification.

23. The method of claim 17, said overlying and underlying images both being hardcopy images, the overlying image being imprinted upon a translucent medium to allow at least partial visibility of the underlying image, said overlaying being performed by direct hand control of the hardcopy images.

24. The method of claim 23, said underlying image also being imprinted on a translucent medium, said underlying and overlying images being displayed on a lightbox.

25. The method of claim 17, said underlying image being displayed by backprojection onto a lightbox, said overlying image being imprinted upon a translucent medium and manually placed on said lightbox.

26. The method of claim 17, wherein said manual control is implemented using a control device selected from the group consisting of:

joystick, trackball, computer mouse, and touchscreen.

27. The method of claim 17, said overlying and underlying images both being displayed on a computer monitor, said overlaying comprising pixelwise mixing of said overlying and underlying images according to a predetermined mixing algorithm.

28. The method of claim 27, wherein said manual control is implemented using a control device selected from the group consisting of:

joystick, trackball, computer mouse, and touchscreen.

29. The method of claim 27, wherein said predetermined mixing algorithm comprises at least one mixing factor that is dynamically user-adjustable, user adjustment of said mixing factor also facilitating user perception of the breast structures.

30. The method of claim 16, said x-ray mammogram image corresponding to a standard x-ray mammogram view plane, said ultrasound image being a thick-slice image representing sonographic properties of a slab-like volume of the breast substantially parallel to said standard x-ray mammogram view plane.

31. The method of claim 30, wherein said slab-like volume occupies less than an entire volume of the breast.

32. The method of claim 31, said thick-slice image being displayed on an electronic display device, further comprising dynamically varying a thickness of said slab-like volume responsive to a first user control input.

33. The method of claim 31, further comprising dynamically varying an elevation of said slab-like volume relative to a reference plane parallel to said standard x-ray mammogram view plane responsive to a second user control input.

34. An apparatus for facilitating analysis of a breast using an x-ray mammogram image thereof and ultrasound information derived from ultrasonic scans of the breast, comprising:

a display device; and

a processor coupled with said display device and generating for viewing thereon a display output, said display output comprising a superposition of a thick-slice ultrasound image and the x-ray mammogram image, said thick-slice ultrasound image representing sonographic properties of a slab-like volume within the breast occupying less than the entire breast volume.

35. The apparatus of claim 34, said x-ray mammogram image corresponding to a standard x-ray mammogram view plane, said slab-like volume being substantially parallel to said standard x-ray mammogram view plane.

36. The apparatus of claim 35, further comprising an input device coupled with said processor for receiving user inputs, said processor dynamically varying a thickness of the slab-like volume responsive to a first user input.

37. The apparatus of claim 36, said processor dynamically varying an elevation of the slab-like volume responsive to a second user input.

38. The apparatus of claim 34, further comprising an input device coupled with said processor for receiving user inputs, said superposition of the thick-slice ultrasound image and the x-ray mammogram image being characterized by an alignment amount ranging from completely non-overlapping to perfectly registered, said processor executing a coarse alignment procedure to change said alignment amount from completely non-overlapping to at least approximately registered, said processor executing a vernier alignment procedure subsequent to said course alignment procedure under direct manual control of the user, user observation of said display output during said vernier alignment procedure facilitating user perception of breast structures.

39. The apparatus of claim 38, said input device comprising a direct manual control input for receiving said direct manual controls, said direct manual control input being selected from the group consisting of:

joystick, trackball, computer mouse, and touchscreen.

40. The apparatus of claim 38, wherein said processor performs said coarse alignment procedure according to an automated registration method.

41. The apparatus of claim 38, wherein said processor performs said coarse alignment procedure under direct manual control of the user.

42. The apparatus of claim 34, further comprising an input device coupled with said processor for receiving user inputs, said superposition being generated according to a predetermined pixelwise mixing algorithm including a mixing factor that is dynamically user adjustable according to a third user input, said user adjustment of said mixing factor also facilitating user perception of the breast structures.

43. An apparatus for facilitating analysis of a breast using an x-ray mammogram image thereof and ultrasound information derived from ultrasonic scans of the breast, comprising:

a display device viewable by a user;

an input device; and

a processor coupled with said input device and with said display device, said processor generating and providing a display output to said display device, said display output comprising a superposition of the x-ray mammogram image and a two-dimensional ultrasound image derived from the ultrasound information, said superposition being characterized by an alignment amount, said processor adjusting said alignment amount according to direct manual manipulations of said input device by the user, user observation of said display output during said alignment adjustments facilitating user perception of breast structures.

44. The apparatus of claim 43, said input device being selected from the group consisting of:

joystick, trackball, computer mouse, and touchscreen.

45. The apparatus of claim 43, said ultrasound image being a thick-slice ultrasound image representing sonographic properties of a slab-like region within the breast occupying less than the entire breast volume.

46. The apparatus of claim 45, said x-ray mammogram image corresponding to a standard x-ray mammogram view plane, said slab-like region being substantially parallel to said standard x-ray mammogram view plane.

47. The apparatus of claim 46, said processor dynamically varying a thickness of the slab-like region responsive to a first user input.

48. The apparatus of claim 46, said processor dynamically varying an elevation of the slab-like volume responsive to a second user input.

49. The apparatus of claim 46, said superposition being generated according to a predetermined pixelwise mixing algorithm.

50. The apparatus of claim 49, said mixing algorithm including a mixing factor that is dynamically user adjustable according to a third user input, said user adjustment of said mixing factor also facilitating user perception of breast structures.