Segmentation of large objects from multiple three-dimensional views

• US 10,631,829 B2
• Filed: 02/27/2014
• Issued: 04/28/2020
• Est. Priority Date: 02/28/2013
• Status: Active Grant

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Dufour, Cecile, Prevost, Raphael Manua Michel, Mory, Benoit Jean-Dominique Bertrand Maurice, Ardon, Roberto Jose
• Primary Examiner(s): Hoffa, Angela M
• Application Number:
  US14/770,483
• Publication Number:
  US 20160007970A1
• Time in Patent Office: 2,252 Days
• CPC Class Codes: A61B 6/5247 combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound
• CPC Class Codes: A61B 8/085 for locating body or organic structures, e.g. tumours, calculi, blood vessels, nodules
• CPC Class Codes: A61B 8/13 Tomography A61B8/10, A61B8/12 take precedence; tomography for radiation diagnosis A61B6/02
• CPC Class Codes: A61B 8/4245 involving determining the position of the probe, e.g. with respect to an external reference frame or to the patient
• CPC Class Codes: A61B 8/4254 using sensors mounted on the probe
• CPC Class Codes: A61B 8/4263 using sensors not mounted on the probe, e.g. mounted on an external reference frame
• CPC Class Codes: A61B 8/461 Displaying means of special interest
• CPC Class Codes: A61B 8/467 characterised by special input means
• CPC Class Codes: A61B 8/483 involving the acquisition of a 3D volume of data
• CPC Class Codes: A61B 8/5207 involving processing of raw data to produce diagnostic data, e.g. for generating an image reconstruction G06T11/00
• CPC Class Codes: A61B 8/5223 for extracting a diagnostic or physiological parameter from medical diagnostic data for algorithms to analyse biomedical images G06T7/0012
• CPC Class Codes: A61B 8/523 for generating planar views from image data in a user selectable plane not corresponding to the acquisition plane
• CPC Class Codes: A61B 8/5238 for combining image data of patient, e.g. merging several images from different acquisition modes into one image for image registration algorithms G06T7/30
• CPC Class Codes: A61B 8/5261 combining images from different diagnostic modalities, e.g. ultrasound and X-ray
• CPC Class Codes: A61B 8/58 Testing, adjusting or calibrating the diagnostic device
• CPC Class Codes: G06T 2207/10136 3D ultrasound image
• CPC Class Codes: G06T 2207/30048 Heart; Cardiac
• CPC Class Codes: G06T 7/149 involving deformable models, e.g. active contour models
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration

Abstract

The present invention relates to an ultrasound imaging system (10) for inspecting an object (33) in a volume (32). The ultrasound imaging system comprises an ultrasound image acquisition probe (14) for acquiring three-dimensional ultrasound images and providing three-dimensional ultrasound image data, comprising a tracking device (25, 27) for tracking a position of the ultrasound image acquisition probe (14) and providing a viewpoint position (128, 130) of the three-dimensional ultrasound images. By this, an improved initialization and improved co-registration and co-segmentation is enabled by providing a plurality of three-dimensional ultrasound images and their respective viewpoint positions (128, 130), and to conduct a segmentation (80) of the object (33) simultaneously out of the plurality of three-dimensional ultrasound images and taking into account the viewpoint positions (128, 130).

Claims

1. An ultrasound imaging system for inspecting an object in a volume, the ultrasound imaging system comprising:

an ultrasound image acquisition probe for acquiring three-dimensional ultrasound images of a volume, wherein the ultrasound image acquisition probe comprises a tracking device for tracking a position of the ultrasound image acquisition probe and providing respective viewpoint positions of the three-dimensional ultrasound images, and

an image processor configured to;

receive a plurality of three-dimensional ultrasound images and the respective viewpoint positions of the plurality of three-dimensional ultrasound images;

simultaneously segment an object out of the plurality of three-dimensional ultrasound images by minimizing an energy term of a space transformation of the object, wherein the energy term is a function of a regularization of the space transformation;

register, concurrently with segmenting the object, the plurality of three-dimensional ultrasound images with one another; and

provide display data corresponding to an image of the three-dimensional ultrasound images aligned based on segmenting the object and registering the plurality of three-dimensional ultrasound images.

2. The ultrasound imaging system of claim 1, wherein the energy term comprises a first term representing a first three-dimensional ultrasound image from the plurality of three-dimensional ultrasound images and at least one further term representing a further three-dimensional ultrasound image from the plurality of three-dimensional ultrasound images, and wherein segmentation of the object further comprises deforming an initial geometric shape such that a deformed initial geometric shape matches a shape of the object, the deformed initial geometric shape being the same in both the first and the at least one further term, wherein deforming the initial geometric shape to the deformed initial geometric shape is defined by the space transformation.

3. The ultrasound imaging system of claim 2, wherein a deformation of the initial geometric shape is conducted by applying a global transformation and a non-rigid local transformation on the initial geometric shape, and wherein the non-rigid local transformation applies a displacement field on the initial geometric shape, wherein a dot product of the global transformation and the non-rigid local transformation define the space transformation.

4. The ultrasound imaging system of claim 3, wherein the applying a global transformation includes at least one of translating, rotating, and scaling the initial geometric shape, and wherein the applying a non-rigid local transformation includes applying a displacement field on the initial geometric shape.

5. The ultrasound imaging system of claim 2, wherein the image processor is configured to determine the initial geometric shape based on a segmentation of the object in three-dimensional image data acquired via a modality other than ultrasound.

6. The ultrasound imaging system of claim 5, wherein the modality is computer tomography.

7. The ultrasound imaging system of claim 2, wherein the image processor is configured to receive the initial geometric shape responsive to user input.

8. The ultrasound imaging system of claim 2, wherein at least one of the first and the at least one further term comprises a registering transformation registering the three-dimensional ultrasound image and the at least one further three-dimensional ultrasound image to a common frame of reference.

9. The ultrasound imaging system of claim 8, wherein the common frame of reference system is defined, at least in part, by a coordinate system associated with one of the initial geometric shape, at least one of the plurality of three-dimensional ultrasound images, or image data obtained from a modality other than an ultrasound.

10. The ultrasound imaging system of claim 1, wherein the image processor is further configured to segment the object by initializing a registration search space, the initializing including positioning an initial geometric shape in each of the plurality of three-dimensional ultrasound images.

11. The ultrasound imaging system of claim 10, wherein the image processor is further configured to register the three dimensional-images to each other by optimizing a three-dimensional translation transformation while taking into account a viewpoint position of the ultrasound image acquisition probe when acquiring a respective three-dimensional image.

12. The ultrasound imaging system of claim 10, wherein the ultrasound imaging system is further configured to refine segmentation and registering transformation performed with respect to the plurality of images by segmenting the object out of another plurality of three-dimensional ultrasound images acquired via the ultrasound image acquisition probe.

13. The ultrasound imaging system of claim 1, wherein the tracking device is an electromagnetic tracking device.

14. A method for providing an ultrasound image of an object in a volume, the method comprising:

providing ultrasound image data comprising a plurality of three-dimensional ultrasound images of the volume and viewpoint positions of each three-dimensional ultrasound image of the plurality of three-dimensional ultrasound images,

processing the ultrasound image data to;

segment the object out of a first ultrasound image of the plurality of three-dimensional ultrasound images, and simultaneously segment the object out of at least one further ultrasound image of the plurality of three-dimensional ultrasound images, by selecting an initial geometric shape approximate in shape to the object, deforming, based on a space transformation, the initial geometric shape to match a shape of the object, wherein an energy term of the space transformation is minimized, wherein the energy term is a function of a regularization of the space transformation, and

concurrently, in parallel with segmenting the object out of the first ultrasound image and the further ultrasound image, register the first ultrasound image with the further ultrasound image to produce a segmented and registered ultrasound image, and

displaying the segmented and registered ultrasound image.

15. The method of claim 14, further comprising inputting a three-dimensional image of the object acquired using a modality other than ultrasound, wherein the three-dimensional image of the object is used as a reference image during registration of the first ultrasound image and the further ultrasound image.

16. The method of claim 15, further comprising calibrating a tracking device for acquiring the viewpoint positions by moving an image acquisition probe coupled to the tracking device to at least two different locations on a known axis of a patient or orienting the image acquisition probe in a predefined relationship to the known axis.

17. A non-transitory computer-readable medium comprising computer-executable instructions for causing a computer including a display to carry out the steps of the method as claimed in claim 14 when said computer-executable-instructions are carried out on the computer.