Method and device of synthesizing panorama image from ultrasound images

• US 6,328,693 B1
• Filed: 01/27/2000
• Issued: 12/11/2001
• Est. Priority Date: 01/28/1999
• Status: Expired due to Term

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Miyatake, Takafumi, Nagasaka, Akio, Umemura, Shinichiro
• Primary Examiner(s): Lateef, Marvin M.
• Assistant Examiner(s): Imam, Ali M.
• Application Number:
  US09/492,132
• Time in Patent Office: 684 Days
• Field of Search: 600/437, 600/443, 600/447, 600/454-456, 600/407, 736/25, 736/26
• US Class Current: 600/437
• CPC Class Codes: A61B 8/14 Echo-tomography
• CPC Class Codes: A61B 8/463 characterised by displaying multiple images or images and diagnostic data on one display composite display in devices for short-range imaging with acoustic waves G01S7/52074
• CPC Class Codes: A61B 8/5238 for combining image data of patient, e.g. merging several images from different acquisition modes into one image for image registration algorithms G06T7/30
• CPC Class Codes: G01S 7/52065 Compound scan display, e.g. panoramic imaging
• CPC Class Codes: G01S 7/52074 Composite displays, e.g. split-screen displays; Combination of multiple images or of images and alphanumeric tabular information
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration

Abstract

A cheap, very operable ultrasound diagnosing device capable of forming a panorama slice image in a real time manner is realized. The device comprises: an imaging unit 120 for converting ultrasound data obtained from a probe 200 into an image; computers (130, 140) having a shift amount detecting function of detecting a shift amount between formed images by the operation of the probe 200 and a rotation amount detecting function of detecting a rotation amount between slice images generated by the operation of the probe 200; and panorama image displaying means 150 for displaying a panorama image 151 obtained by connecting the sequence of slice images captured during diagnosis in a plane manner in accordance with the shift amount and the rotation amount detected by the computers in parallel to the motion image display. Thus, the panorama image can be formed in a real time manner from the sequence of slice images obtained by the probe. The panorama image forming state can be recognized on a display momentarily and an erroneous probe operation can be corrected.

Claims

1. A method of synthesizing a panorama image from ultrasonic images, comprising:

a step of obtaining data of a plurality of consecutive ultrasound frame images constructing an ultrasound motion image;

a step of obtaining a shift amount of a specific area of the plurality of consecutive ultrasound frame images by use of correlation of a characteristic amount derived from said specific area of the plurality of consecutive ultrasound frame images;

a step of obtaining a rotation amount of the specific area of the consecutive ultrasound frame images by use of distributions of shift amounts, each of which is derived from a difference between the characteristic amount of corresponding partial areas obtained by dividing each of the consecutive ultrasound frame images in one axis direction;

a step of detecting positional relation data between the plurality of consecutive ultrasound frame images by use of the shift amount and rotation amount; and

a step of connecting the plurality of ultrasound frame images by using the positional relation data to synthesize panorama image data.

2. The method of synthesizing a panorama image from ultrasound images according to claim 1, further comprising a step of displaying the panorama image data on a display of a display device.

3. The method of synthesizing a panorama image from ultrasound images according to claim 1, wherein said characteristic amount is a projective distribution.

4. An apparatus for synthesizing a panorama image from ultrasound images by obtaining a sequence of a plurality of ultrasound frame images constructing an ultrasound motion image, detecting positional relation data between the plurality of frame images, and synthesizing panorama image data by using the positional relation data, comprising:

an input unit of image data of the plurality of frame images;

a calculating unit including a first signal processing means for obtaining a shift amount of a specific area of the plurality of consecutive ultrasound frame images by use of correlation of a characteristic amount derived from said specific area of the plurality of consecutive ultrasound frame images, and a second signal processing means for obtaining a rotation amount of the specific area of the consecutive ultrasound frame images by use of distributions of shift amounts, each of which is derived from a difference between the characteristic amount of corresponding specific partial areas obtained by dividing each of the consecutive ultrasound frame images in one axis direction; and

means for synthesizing a panorama image from a plurality of frame images by using the shift amount and the rotation amount obtained by the calculating unit.

5. The apparatus for synthesizing a panorama image from ultrasound images according to claim 4, wherein said characteristic amount is projective distribution.

6. The apparatus for synthesizing a panorama image from ultrasound images according to claim 5, wherein each of the first and second signal processing means normalizes the projective distributions in the specific area and the plurality of specific partial areas by using a number obtained by integrating pixels in one axis direction.

7. The apparatus for synthesizing a panorama image from ultrasound images according to claim 6, wherein the pixel used for obtaining the projective distribution is a pixel of a specific color signal.

8. The apparatus for synthesizing a panorama image from ultrasound images according to claim 4, wherein the second signal processing means uses the shift amount obtained from the set of areas which are most apart from each other in the plurality of specific partial areas divided in one axis direction.

9. The apparatus for synthesizing a panorama image from ultrasound images according to claim 4, further comprising image displaying means for converting the motion image data and panorama image data into image display signals and displaying the signals on a display.

10. The apparatus for synthesizing a panorama image from ultrasound images according to claim 9, wherein the image displaying means displays an image process area as a figure for detecting at least one of the shift amount and the rotation amount in a motion image display area on the display.

11. The apparatus for synthesizing a panorama image from ultrasound images according to claim 9, wherein the image displaying means has process area designating means for displaying an image process area as a figure used to form the panorama image in a motion image display area on the display.

12. The apparatus for synthesizing a panorama image from ultrasound images according to claim 9, wherein the image displaying means has position designating means for displaying position information as a figure of a motion image which is currently being processed in the panorama image display area on the display.

13. The apparatus for synthesizing a panorama image from ultrasound images according to claim 9, wherein the image displaying means has means for returning the panorama image to a state at an arbitrary prior time by moving the position information.

14. The apparatus for synthesizing a panorama image from ultrasound images according to claim 4, wherein the input unit of data of the plurality of frame images is connected to a probe for obtaining ultrasound data, and the calculating unit, synthesizing means, and displaying means are constructed to display at least one of a motion image or the panorama image on the display simultaneously with the probe operation.

15. The apparatus for synthesizing a panorama image from ultrasound images according to claim 14, wherein the display is integrally formed with the probe.

16. The apparatus for synthesizing a panorama image from ultrasound images according to claim 14, wherein the probe is provided with means for instructing setup or reset of the probe operation.

17. The apparatus for synthesizing a panorama image from ultrasound images according to claim 4, further comprising detachable image storing means for storing the panorama image data.

18. An apparatus for synthesizing a panorama image from ultrasound images by obtaining data of a sequence of a plurality of ultrasound frame images, detecting positional relation data between the plurality of frame images, and synthesizing panorama image data by using the positional relation data, comprising:

an input unit of image data of the plurality of frame images;

a calculating unit including a first signal processing means for calculating a shift amount and a rotation amount of images in a specific area in the plurality of frame images from the image data;

a step of obtaining a shift amount of a specific area of the plurality of consecutive ultrasound frame images by use of correlation of a characteristic amount derived from said specific area of the plurality of consecutive ultrasound frame images, and a second signal processing means for obtaining a rotation amount of the specific area of the consecutive ultrasound frame images by use of distributions of shift amounts, each of which is derived from a difference between a characteristic amount of corresponding partial areas obtained by dividing each of the consecutive ultrasound frame images in one axis direction;

means for synthesizing a panorama image from the plurality of frame images by using the shift amount and the rotation amount obtained by the calculating unit; and

a display for displaying the motion image and the panorama image by converting the motion image data and the panorama image data into image display signals, wherein an image operation type symbol display area having areas of a panorama instruction button, a reset button, and a forward/backward instruction button is provided in the display, and start of display, interruption, restart, or reversion of the panorama image is performed by the corresponding button in the instruction button area.

19. A recording medium which can be read by a computer and on which a program for obtaining a plurality of frame images constructing an ultrasound motion image, detecting positional relation data of the plurality of frame images, and connecting the plurality of frame images by using the positional relation data, thereby obtaining panorama image data, is recorded, wherein the program comprises:

a first step of calculating a projective distribution of pixels in the vertical direction in a specified area in each of present and previous two frame images from which the positional relation data is to be obtained, and obtaining a shift amount in the horizontal direction between the present and previous two frame images by matching the present and previous projective distributions;

a second step of providing a plurality of divided designated areas in corresponding positions in each of the present and previous two image frames from which the positional relation data is to be obtained, calculating the projective distribution of pixels in each of the plurality of divided designated areas, and matching the projective distributions in the plurality of divided designated areas, thereby obtaining a shift amount in each of the plurality of divided designated areas between the present and previous two frame images;

a third step of calculating a rotational angle between the present and previous two frame images by using the shift amount of each of the divided designated areas obtained in the second step; and

a fourth step of shifting and rotating the image area to be synthesized in the present image frame by the shift amount obtained in the first step and the rotational angle obtained in the third step, and writing the result into a panorama image memory, the first to fourth steps being repeatedly executed on a newly input image frame as the present image frame.

20. The recording medium which can be read by a computer, according to claim 19, wherein in the second step, the ratio of effective pixels to all pixels in each of the plurality of divided partial areas is checked and, when the ratio is equal to or larger than a predetermined value, the projective distribution of the divided designated area is calculated, and the third step has a step of selecting the shift amount between two divided designated areas which are most apart from each other among the shift amounts of the plurality of divided partial areas, and a step of performing a calculation of dΘ

=tan¹ (dxa−

dxb)/h, where each of dxa and dxb denotes a shift amount between the two divided designated areas and dΘ

denotes the rotational angle.