Quantitative analysis, visualization and movement correction in dynamic processes

• US 20060034545A1
• Filed: 03/08/2002
• Published: 02/16/2006
• Est. Priority Date: 03/08/2001
• Status: Active Grant

• Current Assignee: Johannes Fieres, Joseph Fourier University, Julian Mattes, Roland Eils
• Original Assignee: Universite Joseph Fourier
• Inventors: Mattes, Julian, Fieres, Johannes, Demon-Geot, Jacques, Eils, Roland
• Granted Patent: US 7,590,264 B2
• Time in Patent Office: Days
• US Class Current: 382/294
• CPC Class Codes: G06T 2207/30008 Bone
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

The invention relates to the quantitative analysis and/or visualization (virtual or real) of moving processes and to the detection, description, correction and the comparison of global movements inside the image space. The invention particularly relates to a method and device for precisely and, while being limited to few parameters, quantitatively describing the global and local movement occurring in the image space and for the single representation of the movement of the quantitative parameters with regard to the image space. A rough to fine recording and a detection of and compensation for global movements occurs whereby enabling a tracking and a corrected visualization. The detection of the movement, which corresponds as precisely as possible to the real movement occurring in the image space is effected by a splines a plaques minces recording technique that enables the number of movement parameters to be kept low. A fully automatic execution of all partial steps is made possible.

Claims

1. A method for the quantitative and visual analysis of motion in a time sequence of images comprising the steps of:

a) determination of the motion vectors for each point in the image space between images of directly or not directly subsequent time steps (i.e., the transformation of the corresponding image space) preferably by means of rigid, affine and/or “

elastic”

(including local deformations) overlapping/registration of images of an image sequence and calculation of the transformation parameter comprising the following further method steps;

a1) for the non-rigid registration, local transformation types are given stepwise from coarse to fine, first global and then increasingly more local;

a2) for more then two time steps, in order to obtain the transformation for not directly subsequent timesteps, the transformation is not only calculated in pairs for subsequent image pairs, but also a correction-registration step is further added thereto;

a3) from the transformation result directly the local, defined for each point in the image/time space, quantities velocity, acceleration, warp, tension, deviation from the finest registration (i.e., distance between the point transformed by the selected transformation type and by the most local fineness level), etc. as well as the global quantities warp energy for the entire motion as well as for each “

principal warp”

separately (Bookstein

     1989), entropy, “

directedness”

of the motion and the transformation parameters for various fineness levels of the transformation;

b) reconstruction of the space-time-structure within the image space and visualization thereof by means of

b1) (optionally interactive) determination of a point (or a region;

or according to a regular scheme selected point in the space or on the surface of a region) for a selectable moment, the automatic calculation of its (their) position for all other moments by means of the under a) calculated transformation (for not directly subsequent time steps) as well as the interpolated representation as 3D rendered (surface rendering) path (or “

tube”

for regions); and

/or

b2) a 3D rendered reference grid, deformed according to the transformation calculated in a), animated (interpolation of intermediate time steps) or not (animation also continuously over several time steps), the image space overlapped (inserted) or not;

in an image record having several color channels, the reference grids corresponding to the various separately treated color channels can be represented in different color but simultaneously; and

/or

b3) color/pattern encoding (including gray value encoding) of quantities (quantities having vector values by means of absolute values), which are assigned to a point in the image/time space as described under a), particularly for (i) all points, lying within an interactively selectable level of the image space, (ii) interactively (or regularly) selected points or surface points of interactively selected regions/shapes/surfaces (iii) the points of the under b1) determined path (or tubes) of the time space; and

/or

b4) a motion corrected representation of the paths (or tubes), corrected by a rigid, affine or a selected level of the non-global transformation, or by means of a motion corrected playing of the original image sequence, particularly the color encoding can be back projected onto the original image or any other given moment.

2. The method according to claim 1, which extracts in a pre-processing step a0) critical (characteristic) points of the image space or surface points of regions in the image space and uses these in the steps b1), b3) and b4) instead of interactively selected points, whereby in step b2) the (critical/surface) points of the original space can be used instead of the reference grid.

3. The method according to claim 2, which uses for the preprocessing the confinement tree technique (CTT), whereby

a) for the visualization of a region (a confiner), the user clicks either on knot in the tree or on a point in the image, upon which the surface/shape of the smallest confiners (in the filtered tree) containing said point will be shown;

and/or

b) the registration is performed on the basis of the extracted points (confiner-surface points or critical points, e.g., confiner center points) by means of a point registration algorithm, whereby confiners corresponding to each other are pre-determined in the original and target image or an algorithm based on the minimization of an “

iconic”

similarity measure follow; and

/or

c) transformed confiner of the original image overlapped with non-transformed confiners of the target image can be represented whereby the confiner surfaces replace the parts of the reference grid in 1b2), and to the quantities from 1a3) the distance from one point to the next adjacent point extracted in the other image is added.

4. A method for registration by means of CTT, which either

a) uses confiner center points (CS) or confiner density maxima (CD) as critical points for registration and by using the method of structural outlier determination (by means of relPos, see Mattes and Demongeot 2001) starting from an initial overlapping (optionally improved by a direct use of a point registration method, see page 6) ) first determines corresponding confiner pairs and thereby only maintains pairs with best relPos (optionally according to 6(i)), on the basis of this (CD, CS or confiner shapes of which) determines the best fitting (in the sense of the error functional, see 6) as well as Fieres et al. 2001) transformation, exerts on the data confiner and repeats this process iteratively, whereby the such determined landmarks can be used for the establishment of a “

point distribution model”

;

or

b) corresponding confiners are determined as follows;

for the knots, which occur after a bifurcation in the CT of one of the images, the knots/confiner having the lowest relPos value in the CT of the other image are searched, the point outliers are thereby suppressed as in claim 6, then the relPos distribution for all confiner pairs are examined, and the confiner outlier are eliminated temporarily as described in 6)(i) for point outliers,

finally a point registration method under consideration of corresponding confiners are used,

by an optional usage of first highly cut-down CTs and then increasingly less highly cut-down CTs a coarse to fine effect can be achieved in a) and b), which can also be combined with/replaced by the following method(s) and the images are first highly (i.e., on great scales), (e.g., Gauss-) smoothed (or presented in a high pixel resolution), and in the proceeding of the method increasingly less highly smoothed (or represented in increasingly finer resolution).

5. The method according to claim 3 using the CTT as in claim 4.

6. The method according to claim 3, which instead of the pre-processing by means of CTT identifies objects in the image space in a segmentation step and calculates the surface and/or center point of which, the objects replace the confiners, in case only (non-coherent) points are identified on the object surface (e.g., by means of a Canny edge-detector), corresponding confiner/objects cannot be pre-determined according to 3b) and particularly for the segmented objects can also individual translations and rotation or deformations be determined and the object paths can individually corrected by selected transformation types.

8. The method according to claim 3, 4 and/or 6, using the method of claim 7 for point registration.

9. The method according to claim 1 to 3, 4, 5, 6 and/or 8, that automatically (also partially) compares the transformation determined in ia) (or corresponding) with idealized dynamic reference types (in time and space) as, for example, spatial linear dynamic, diffusion dynamic, Hamilton dynamic, elastic/harmonic vibration etc., whereby this is performed by the steps:

0 search for the best fitting reference dynamic und determination of its parameters and of the pointwise and absolute error with reference to the transformation;

1 optionally alternatively to 1b);

color/pattern encoded representation of the (pointwise) error at the surface/in the image volume.

12. Computer program comprising a program code means for performing a method according to any one of the preceding claims, if the computer program is executed by a computer.

13. A computer program product comprising a program code means which is stored on a computer readable medium, in order to perform a method according to any one of claims 1 to 11, if the program product is executed by a computer.

14. A data processing system particularly for performing the method according to any one of claims 1 to 11.

7. A method for the registration of a data point (DP) set on a reference point (RP) set, which determines iteratively for a series of given transformation types, which allow an increasing number of local deformations, its free parameters in such a manner, that an error functional is minimized, which compares the positions of the transformed data points with those of the reference points, thereby selecting as transformation types first a rigid (rotation and translation), then an affine transformation and then transformation types, defined by a number of checkpoints, which can freely be displaced (the free parameter of the transformation) and between the displacement vectors of which with thin-plate splines is interpolated (Bookstein 1989) and, the number of checkpoints is iteratively increased (in order to allow an increasing number of local deformation), and the initial position of the checkpoints is adapted to the form of the data points and/or the reference points or the relative position thereof,

particularly for the determination of the initial position of the data point set—

whereby each point can be weight depending on its distance to the next reference point—

a point-lustering-algorithm can be exerted und the cluster center points (or the point with the highest density within each cluster, etc.) can be selected as the initial position,

particularly also the sum of the quadratic distances of each DP to the next RP (to this term the distances of each reference point to the next data point can be added and/or a regularization term which smoothes too high warp) can be selected as error functional,

outliers can be suppressed in several ways;

(i) starting from the distribution of the distances between the data points and the next reference point, all data points/reference points having higher distances as the median or average+standard deviation of the distribution are eliminated or (ii) instead of the quadratic distances another function of the distance is used (e.g., amount function or see Miesmator in Press et al.

     1992).

10. A method for the partition of the space into regions of different, internally homogeneous motion, whereby the dynamic parameters are separately determined for the various regions and for this purpose first for the point set of a series of time steps a “

point distribution model”

is established, for which the dominating main components of motion are selected, and then each point of the space is coordinated to the component which contributes the dominating part to its displacement during the time step under examination, the various region are represented by different colors.

11. A method for the continuous numerical description of the motion by means of statistic techniques, like clustering and histogram analysis, which simplifies the comparison of motions, application of these techniques on the pointwise determined local quantities (e.g. on the in 1b3) (i) selected level), in order to mark regions, in which a certain quantity occurs especially intensive or in which particularly those values occur, which are assumed especially often (clustering on the basis of the histogram).