Ultrasound imaging with real time 3D image reconstruction and visualization

• US 5,787,889 A
• Filed: 02/21/1997
• Issued: 08/04/1998
• Est. Priority Date: 12/18/1996
• Status: Expired due to Term

• Current Assignee: University of Washington
• Original Assignee: University of Washington
• Inventors: Edwards, Warren, Deforge, Christian, Kim, Yongmin
• Primary Examiner(s): JAWORSKI, FRANCIS J
• Application Number:
  US08/804,054
• Time in Patent Office: 529 Days
• Field of Search: 128/660.04-660.05, 128/660.07, 128/661.01, 128/916, 395/119, 382/128, 382/285
• US Class Current: 600/443
• CPC Class Codes: A61B 8/06 Measuring blood flow measuring volume flow in general G01F, e.g. G01F1/66, G01F1/72; measuring speed of fluids in general G01P5/00
• CPC Class Codes: A61B 8/08 Detecting organic movements or changes, e.g. tumours, cysts, swellings A61B8/02 - A61B8/06 take precedence
• CPC Class Codes: A61B 8/13 Tomography A61B8/10, A61B8/12 take precedence; tomography for radiation diagnosis A61B6/02
• CPC Class Codes: A61B 8/14 Echo-tomography
• CPC Class Codes: A61B 8/483 involving the acquisition of a 3D volume of data
• CPC Class Codes: G01S 15/8993 Three dimensional imaging systems
• CPC Class Codes: G01S 7/52034 Data rate converters
• CPC Class Codes: G01S 7/52044 Scan converters
• CPC Class Codes: Y10S 128/916 Ultrasound 3-D imaging

Abstract

A 3D image is generated in real-time on an ultrasound medical imaging system which performs acquisition, volume reconstruction, and image visualization tasks using multiple processors. The acquisition task includes deriving position and orientation indicators for each gathered image frame. Volume reconstruction includes defining a reference coordinate system within which each image frame in a sequence of image frames is registered. The reference coordinate system is the coordinate system for a 3D volume encompassing the image planes to be used in the 3D image. The first image frame is used to define the reference coordinate system. As each image plane is registered, a 2D projection of the incremental volume is displayed. A shear-warp factorization process is used to derive a 2D projection for a rotated volume. A viewing transformation matrix is factorized into a 3D shear which is parallel to slices of the reference volume. A 2D warp then is implemented to produce the projection of the rotated volume.

Claims

1. A method of generating an ultrasound image of a three-dimensional target volume from a plurality of ultrasound data frames corresponding to a scan of the target volume, each frame comprising data for a plurality of sample points within an ultrasound image plane encompassing said plurality of sample points, the method comprising the steps of:

for a first frame of the plurality of frames and a corresponding first image plane, defining a first image plane coordinate system for the sample points of said first frame, the first image plane coordinate system having a first axis and a second axis orthogonal to the first axis;

defining a reference coordinate system having three orthogonal axes, in which a first reference coordinate system axis is parallel to the first axis of the first image plane coordinate system, a second reference coordinate system axis is parallel to the second axis of the first image plane coordinate system, and a third reference coordinate system axis is perpendicular to the first image plane, and wherein the first image plane occurs where a coordinate of the third reference coordinate system axis is zero;

for each i-th frame of the plurality of ultrasound data frames after the first frame,

(i) identifying an i-th image plane coordinate system for the sample points of said i-th frame;

(ii) transposing i-th image plane coordinate system coordinates of each sample point in the i-th image plane into real-valued coordinates within the reference coordinate system;

(iii) for each one of a plurality of sample points within the i-th image plane interpolating a data value for the sample point to achieve an interpolated value for a discrete-valued coordinate of the reference coordinate system nearest to said sample point'"'"'s real-valued reference coordinate system coordinate, and

(iv) storing the achieved interpolated value in a memory location corresponding to the reference coordinate system discrete-valued coordinate; and

outputting memory contents for the reference coordinate system discrete-valued coordinates to generate a display image projection of at least a three dimensional portion of the target volume.

2. The method of claim 1, further comprising after the step of defining the reference coordinate system, allocating first memory address space for storing data points for discrete coordinate positions for a rectilinear reference volume, the reference volume being defined within a first quadrant of the reference coordinate system, and wherein said steps of interpolating and storing are performed for each one sample point within the i-th image plane occurring within the reference volume, and wherein said memory location for said storing step is within said first memory address space.

3. The method of claim 2, in which the first memory address space is allocated to store achieved interpolated values along a given row of a given third reference coordinate system axis slice of the reference volume in sequential address locations, and wherein the first memory address space is further allocated to store achieved interpolated values for adjacent rows of the given third reference coordinate system axis slice in adjacent first memory address space.

4. The method of claim 1, further comprising the step of generating pixel data for the display image projection, the step of generating pixel data comprising for a given pixel the step of identifying reference coordinate system discrete-valued coordinates which overlap for the display image projection and using only one of the achieved interpolated values for the overlapping coordinates to derive the pixel data for the given pixel.

5. The method of claim 1, further comprising the step of generating a display image projection of at least a three dimensional portion of the target volume incrementally after the steps of transposing, interpolating and storing are performed for any i-th image plane.

6. The method of claim 5, in which an operator viewing the target volume portion is able to view gaps in the target volume portion where sample points were not obtained during scanning, and in response re-scanning a portion of the target volume to fill in the gaps.

7. The method of claim 1, wherein for each i-th frame of the plurality of ultrasound data frames, a position sensor senses position and orientation information, the position and orientation information for a given i-th frame being used in the transposing step for transposing the i-th image plane coordinate system coordinates into reference coordinate system coordinates.

8. The method of claim 1, wherein the display image projection of the portion of the target volume is for a first viewing direction of the target volume portion, and further comprising the step of generating a display image projection of the target volume portion for a second viewing direction different than the first viewing direction.

9. The method of claim 8, in which the step of generating comprises:

computing a translation offset for respective slices of the reference volume;

processing at a first processor a first rectangular portion of each one slice of a first plurality of slices within a first overlap region occurring for the second viewing direction, wherein size of the first rectangular portion varies for each one of the first plurality of slices;

processing at a second processor a second rectangular portion of each one slice of a second plurality of slices within a second overlap region occurring for the second viewing direction; and

wherein the step of processing at a first processor comprises determining which stored achieved interpolated value corresponding to overlapping points on the display image projection of the first plurality of slices is displayed for the second viewing direction; and

wherein the step of processing at a second processor comprises determining which stored achieved interpolated value corresponding to overlapping points on the display image projection of the second plurality of slices is displayed for the second viewing direction.

10. The method of claim 9, in which the first rectangular portion of said first plurality of slices comprises multiple row portions of each one slice of said first plurality of slices, each row portion of a said one slice corresponding to a contiguous portion of memory storing the achieved interpolated values for adjacent coordinates within a corresponding row of the reference coordinate system.

11. The method of claim 10, in which the first processor and second processor reside on a common integrated circuit, the common integrate circuit further comprising a transfer controller and on-chip memory, and wherein the transfer controller moves data corresponding to the first rectangular portions on-chip while the first processor processes data corresponding to a rectangular portion other than the first rectangular portion.

12. The method of claim 1, wherein the step of interpolating a data value for a sample point comprises interpolating the data value for said sample point to achieve an interpolated value for each one of a plurality of discrete-valued coordinates of the reference coordinate system nearest to said sample point'"'"'s real-valued reference coordinate system coordinate.

13. A method of incrementally generating an ultrasound image of a three-dimensional target volume in real time from a sequence of ultrasound data frames corresponding to a scan of the target volume, each frame comprising data for a plurality of sample points within an ultrasound image plane encompassing said plurality of sample points, the method comprising the steps of:

for a first frame of the plurality of frames and a corresponding first image plane, defining a first image plane coordinate system for the sample points of said first frame, the first image plane coordinate system having a first axis and a second axis orthogonal to the first axis;

defining a reference coordinate system having three orthogonal axes, in which a first reference coordinate system axis is parallel to the first axis of the first image plane coordinate system, a second reference coordinate system axis is parallel to the second axis of the first image plane coordinate system, and a third reference coordinate system axis is perpendicular to the first image plane, and wherein the first image plane occurs where a coordinate of the third reference coordinate system axis is zero;

for each i-th frame of the plurality of ultrasound data frames after the first frame,

(i) identifying an i-th image plane coordinate system for the sample points of said i-th frame,

(ii) transposing i-th image plane coordinate system coordinates of each sample point in the i-th image plane into real-valued coordinates within the reference coordinate system,

(iii) for each one of a plurality of sample points within the i-th image plane interpolating a data value for the sample point to achieve an interpolated value for a discrete-valued coordinate of the reference coordinate system nearest to said sample points real-valued reference coordinate system coordinate,

(iv) storing the achieved interpolated value in a memory location corresponding to the reference coordinate system discrete-valued coordinate; and

(v) outputting memory contents for the reference coordinate system discrete-valued coordinates to generate a display image projection of at least a three dimensional portion of the target volume.

14. An ultrasound system for generating an ultrasound image of a three-dimensional target volume in real time from a plurality of ultrasound data frames corresponding to a scan of the target volume, each frame comprising data for a plurality of sample points within an ultrasound image plane encompassing said plurality of sample points, the system comprising:

a transducer array for transmitting ultrasound energy into a patient in the vicinity of the target volume and for receiving echoes of the ultrasound energy, the received echoes being transformed into the frames of ultrasound data;

a processing apparatus for processing the frames of ultrasound data to generate image data; and

a display device for displaying an image projection of at least a three dimensional portion of the target volume, wherein the image projection is derived from the image data;

wherein for a first frame of the plurality of frames and a corresponding first image plane, the processing apparatus defines a first image plane coordinate system for the sample points of said first frame, the first image plane coordinate system having a first axis and a second axis orthogonal to the first axis;

wherein the processing apparatus defines a reference coordinate system having three orthogonal axes, in which a first reference coordinate system axis is parallel to the first axis of the first image plane coordinate system, a second reference coordinate system axis is parallel to the second axis of the first image plane coordinate system, and a third reference coordinate system axis is perpendicular to the first image plane, and wherein the first image plane occurs where a coordinate of the third reference coordinate system axis is zero;

wherein for each i-th frame of the plurality of ultrasound data frames after the first frame, the processing apparatus

(i) identifies an i-th image plane coordinate system for the sample points of said i-th frame,

(ii) transposes i-th image plane coordinate system coordinates of each sample point in the i-th image plane into real-valued coordinates within the reference coordinate system,

(iii) for each one of a plurality of sample points within the i-th image plane, interpolates a data value for the sample point to achieve an interpolated value for a discrete-valued coordinate of the reference coordinate system nearest to said sample point'"'"'s real-valued reference coordinate system coordinate,

(iv) and stores the achieved interpolated value in a memory location corresponding to the reference coordinate system discrete-valued coordinate as image data.

15. The system of claim 14, further comprising first memory having an address space for storing data points for discrete coordinate positions for a rectilinear reference volume, the reference volume being defined within a first quadrant of the reference coordinate system, and wherein the processing apparatus interpolates and stores within the first memory or each one sample point within the i-th image plane occurring within the reference volume.

16. The system of claim 15, in which the first memory address space is allocated to store achieved interpolated values along a given row of a given third reference coordinate system axis slice of the reference volume in sequential address locations, and wherein the first memory address space is further allocated to store achieved interpolated values for adjacent rows of the given third reference coordinate system axis slice in adjacent first memory address space.

17. The system of claim 14, in which the processing apparatus generates pixel data for the display image projection by identifying reference coordinate system discrete-valued coordinates which overlap for the display image projection and using only one of the achieved interpolated values for the overlapping coordinates to derive the pixel data for the given pixel.

18. The system of claim 17, in which the target volume portion is constructed and displayed incrementally as respective frames of ultrasound data are processed by the processing apparatus.

19. The system of claim 14, in which a display image projection of at least a three dimensional portion of the target volume is generated and displayed incrementally after the processing apparatus transposes, interpolates, and stores for any i-th image plane.

20. The system of claim 14, wherein the display image projection of the portion of the target volume is for a first viewing direction of the target volume portion, and wherein the processing apparatus generates pixel data for a display image projection of the target volume portion for a second viewing direction different than the first viewing direction.

21. The system of claim 20, in which the processing apparatus comprises a multiprocessor integrated circuit having a first processor, a second processor, a transfer controller and on-chip memory;

the multiprocessor computing a translation offset for respective slices of the reference volume;

the first processor processing a first rectangular portion of each one slice of a first plurality of slices within a first overlap region occurring for the second viewing direction, wherein size of the first rectangular portion varies for each one of the first plurality of slices, and the first processor determining for each set of overlapping points on the display image projection of the first plurality of slices within the first rectangular portion, which stored achieved interpolated value is displayed for the second viewing direction;

the second processor concurrently processing a second rectangular portion of each one slice of a second plurality of slices within a second overlap region occurring for the second viewing direction, and the second processor determining for each set of overlapping points on the display image projection of the second plurality of slices within the second rectangular portion, which stored achieved interpolated value is displayed for the second viewing direction.

22. The system of claim 21, in which the first rectangular portion of said first plurality of slices comprises multiple row portions of each one slice of said first plurality of slices, each row portion of a said one slice corresponding to a contiguous portion of memory storing the achieved interpolated values for adjacent coordinates within a corresponding row of the reference coordinate system.

23. The system of claim 22, wherein the transfer controller moves data corresponding to the first rectangular portions on-chip while the first processor processes data corresponding to a rectangular portion other than the first rectangular portion.