Quantitative analysis, visualization and movement correction in dynamic processes

• US 7,590,264 B2
• Filed: 03/08/2002
• Issued: 09/15/2009
• Est. Priority Date: 03/08/2001
• Status: Expired due to Fees

• Current Assignee: Johannes Fieres, Joseph Fourier University, Julian Mattes, Roland Eils
• Original Assignee: Johannes Fieres, Julian Mattes, Roland Eils, Universite Joseph Fourier
• Inventors: Eils, Roland, Mattes, Julian, Fieres, Johannes, Demon-Geot, Jacques
• Primary Examiner(s): Tabatabai; Abolfazl
• Application Number:
  US10/471,153
• Publication Number:
  US 20060034545A1
• Time in Patent Office: 2,748 Days
• Field of Search: 382/103, 382/107, 382/151, 382/206, 382/294, 348/97, 348/154, 348/155, 700/47, 700/124, 702/23, 702/129, 717/105, 717/109, 717/113, 717/125
• US Class Current: 382/107
• CPC Class Codes: G06T 2207/30008 Bone
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

The invention relates to the quantitative analysis and/or visualization (virtual or real) of moving processes and to the detection, description, correction and the comparison of global movements inside the image space. The invention particularly relates to a method and device for precisely and, while being limited to few parameters, quantitatively describing the global and local movement occurring in the image space and for the single representation of the movement of the quantitative parameters with regard to the image space. A rough to fine recording and a detection of and compensation for global movements occurs whereby enabling a tracking and a corrected visualization. The detection of the movement, which corresponds as precisely as possible to the real movement occurring in the image space is effected by a splines a plaques minces recording technique that enables the number of movement parameters to be kept low. A fully automatic execution of all partial steps is made possible.

Claims

1. A method for the quantitative and visual analysis of motion in a time sequence of images comprising:

using a computer-implemented to perform the following steps;

a) determining the motion vectors for points in the image space between images of subsequent time steps by means of registration of images of an image sequence;

b) reconstruction of the space-time-structure within the image space and visualization thereof by means of at least one of the following steps b1) to b4);

b1) determining a point for a selectable moment, calculating automatically its position for other moments by a means of a transformation resulting from the registration as calculated in a) as well as determining and visualizing the interpolated representation of the path which relates to the determinined point and its calculated transformed positions as 3D rendered path;

b2) determining and visualizing a 3D rendered reference grid, deformed according to the transformation calculated in a), where the grid is optionally displayed successively for intermediate degrees of deformation from an undeformed until the deformation corresponding to the transformation calculated in a) where in a image record having several color channels, the reference grids corresponding to the different color channels are separately treated;

b3) color/pattern encoding (including gray value encoding) of quantities which are assigned to a point in the image/time space and where vector valued quantities are represented by a means of absolute values, for

(i) all points, lying within an interactively selectable plane of the image space, or

(ii) for any selected points or

(iii) the points of the under b1) determinined path of the space; and

b4) a motion corrected representation of the paths or of any other means of visualization described in b2)-b3), where the motion is corrected by undoing the part of the motion which corresponds to a selected class of transformations and which is calculated by registration.

2. The method according to claim 1, which extracts in a pre-processing step points of the image space and uses these in the steps b1), b3) and b4) instead of interactively selected points, whereby in step b2) the extracted points of the original image space can be used instead of the reference grid.

3. The method according to claim 2, which uses for the preprocessing the confinement tree technique (CTT), whereby

the extracted points are surface points of regions in the image and for the visualization of the region, the user clicks the image, upon which the smallest confiners (in the filtered tree) containing said point will be shown.

4. The method according to claim 3, including means identifying objects in the image space in a segmentation step and calculating the surface points and the geometric center of these objects, whereby the objects play the role of confiners in claim 3, and including means determining for the segmented objects can also individual translations, rotation and deformations and correcting the object paths are individually selected transformation types.

5. The method according to claim 2, which uses for the preprocessing the confinement tree technique (CTT), whereby

the registration is preformed on the basis of the extracted points by means of a point registration algorithm, whereby confiners corresponding to each other are pre-determined in the original and target image.

6. The method according to claim 2, which uses for the preprocessing the confinement tree technique (CTT) whereby

transferred confiner of the original image are displayed overlaying with non-transformed confiners of the target image whereby the confiners surfaces replace the parts of the reference grid in b2), and to the quantities from a3) the distance form one point to the next adjacent in the other image is added.

7. The method according to claim 1 including automatically comparing the transformation determinined with idealized dynamic reference types.

8. The method according to claim 1, wherein the registration step is a parametric non-rigid registration, wherein local transformation types are given stepwise from coarse to fine, first global and then increasingly more local.

9. The method according to claim 1, wherein for more then two time steps, the transformation obtained by registration in step a) is calculated in pairs for subsequent image pairs and also a correction-registration step is further added thereto.

10. The method according to claim 1, further comprising the step of determining form the transformation result for each point in the image/time space, the local quantities of velocity, acceleration, bending, tension, deviation from the finest registration as well as at least one of the global quantities of bending energy for the entire motion and as well as for “

principal warp”

separately, entropy, “

directness”

of the motion and transformation parameters for various fineness levels of the transformation.

11. A method implemented by a computer for registration by means of a confinement tree technique CTT, which uses one of,

a) confiner center points (CS) or confiner density maxima (CD) as critical points for registration and by using the method of structural outlier determination starting from an initial overlapping first determines corresponding confiner pairs and thereby only maintains pairs with best relPos on the basis of this determines the best fitting transformation, exerts on the data confiner and repeats this process iteratively, whereby the such determined landmarks can be used for the establishment of a “

point distribution model”

; and

b) corresponding confiners are determined as follows;

for knots, which occur after a bifurcation in the confinement tree (CT) of one of the images, the knots having the lowest relPos value in the CT of the other image are searched, the point outliers are thereby suppressed then the relPos distribution for all confiner pairs are examined, and confiner outliers are eliminated temporarily, whereupon a point registration method under consideration of corresponding confiners is applied to the confiner surface or contour points and this process is repeated iteratively,

by an optional usage of first highly cut-down CTs and then increasingly less highly cut-down CTs a coarse to fine effect can be achieved in a) and b), and wherein the images are first highly smoothed, and in the proceeding of the method increasingly less highly smoothed.

12. A method implemented by a computer for the registration of a data point (DP) set on a reference point (RP) set, which determines iteratively for a series of given transformation types, which allow an increasing number of local deformations, its free parameters in such a manner, that an error functional is minimized, which compares the positions of the transformed data points with those of the reference points, thereby selecting as transformation types first a rigid (rotation and translation), then an affine transformation and then transformation types, defined by a number of checkpoints, which can freely be displaced and between the displacement vectors of which with thin-plate splines is interpolated and, the number of control points is iteratively increased in order to allow an increasing number of local deformation, and the initial position of the control points is adapted to the shape formed of the data points and the reference points or the relative position thereof, particularly for the determination of the initial position of the control points a point-clustering-algorithm is exerted on the data point set, whereby each data point can be weighted depending on its distance to the next reference point, and cluster center are selected as the initial control point positions, and a sum of the quadratic distances of each DP to the next RP is selected as error functional whereby outliers can be suppressed based upon at least of:

(i) starting from the distribution of the distances between the data points and the next reference point, all data points having higher distances as the median of the distribution are eliminated or (ii) instead of the quadratic distances another function of the distance is used.