Method of processing images in order automatically to detect key points situated on the contour of an object and device for implementing this method

• US 5,617,459 A
• Filed: 07/12/1995
• Issued: 04/01/1997
• Est. Priority Date: 07/12/1994
• Status: Expired due to Fees

• Current Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Original Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Inventors: Makram-Ebeid, Sherif, Breitenstein, Jacques
• Primary Examiner(s): Porta, David P.
• Assistant Examiner(s): Bruce, David Vernon
• Application Number:
  US08/500,653
• Time in Patent Office: 629 Days
• Field of Search: 382/128, 382/131, 382/132, 382/156, 395/924, 364/413.13, 378/62, 378/901
• US Class Current: 378/62
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30048 Heart; Cardiac
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling

Abstract

A method of processing images in order automatically to detect key pixels (K₂, K₃, K₄) situated on the contour of an object (LV) in an initial image (I₀), and a device for implementing this method in which there is storage, in the digitized initial image (I₀), of the intensity of the pixels [A₀ (x,y)] and of data of regions of the object (LV), classes (C₁ C_m); selection of pixels of interest (PI), on the contour, inside and outside the object (LV); generation of characteristics (E₁ E_k) for each of the pixels of interest (PI); classification of the pixels of interest (PI) into the classes (C₁ C_m); and selection of the key pixels (K₂, K₃, K₄) from corresponding classes (C₂, C₃, C₄).

Claims

1. Method of processing images in order automatically to detect points situated on the contour of an object (LV) at predetermined structure, referred to as key pixels (K₂, K₃, K₄), in an image referred to as the initial image (I₀), this method comprising a first phase including steps of:

storage in the initial image (I₀) in the form of a two-dimensional matrix of pixels of the intensity values of each pixel (A₀) labelled by its coordinates (x,y);

storage in the initial image (I₀) of data of regions of the object (LV) which are referred to as classes (C₁ to C_m) including classes which are referred to as corresponding classes (C₂, C₃, C₄) respectively containing the key pixels (K₂, K₃, K₄) to be detected;

selection of pixels of the initial image (I₀) which are referred to as pixels of interest (PI), on the contour of and inside and outside the object (LV);

generation of a first vector of characteristics (E₁ to E_k) for each of the pixels of interest (PI); and

classification of the pixels of interest (PI) into the said classes (C₁ to C_m) of the object (LV) on the basis of their respective vector of characteristics (E₁ to E_k).

2. Method of processing images in order automatically to detect points situated on the contour of an object (LV), referred to as key pixels (K₂, K₃, K₄), in an image referred to as the initial image (I₀), this method comprising a first phase including steps of:

storage in the initial image (I₀) in the form of a two-dimensional matrix of matrix of pixels of the intensity values of each pixel (A₀) labelled by its coordinates (x, y);

storage in the initial image (I₀) of data of regions of the object (LV) which referred to as classes (C₁ to C_m) including classes which are referred to as corresponding classes (C₂, C₃, C₄) respectively containing the key pixels (K₂, K₃, K₄) to be detected;

selection of pixels of the initial image (I₀) which are referred to as pixels of interest (PI), on the contour of and inside and outside the object (LV);

generation of a first vector of characteristics (E₁ to E_k) for each of the pixels of interest (PI); and

classification of the pixels of interest (PI) into the said classes (C₁ to C_m) of the object (LV) on the basis of their respective vector of characteristics (E₁ to E_k);

wherein the step of selection of the pixels of interest (PI), sub-steps of;

filtering of the initial image (I₀) in order to select pixels having a gradient of maximum intensity which are situated on the contour of the object (LV); and

sampling of the initial image (I₀) at a lower frequency, preferentially preserving the pixels having a gradient of maximum intensity.

3. Method according to claim 2 furthermore comprising, in the first phase, steps of:

multiresolution filtering of the initial image (I₀), by Gaussian functions, and sub-sampling of the filtered images in order to form a pyramid of Gaussian images (I₀ to I₇);

generation of said first vector of characteristics relating to each pixel of interest (PI) from the image data of this pyramid of Gaussian images.

4. Method according to claim 3, comprising steps of:

log-polar transformation applied to the pixels of interest (PI) in accordance with a chart formed of circular sectors;

generation of said first vector of characteristics by estimating each characteristic (E₁ to E_k) as a mean of the intensities of the pixels in each circular sector of the chart relating to the log-polar transformation, calculated by interpolation of the intensity of four neighboring pixels selected from one of the images of the pyramid of Gaussian images, which coincide with the four corner pixels delimiting said relevant circular sector, while taking account of the distance apart of these pixels, due to the frequency of sampling of the Gaussian image used.

5. Method according to claim 4, comprising a step of:

generation of said first vector of characteristics, with the addition, to the characteristics already calculated, of special characteristics, including the modulus of the gradient of the pixels of interest, the direction in which this gradient is a maximum, and the curvature of the lines of equal grey level.

6. Method according to claim 3, comprising steps of:

calculation of the intensity of each pixel of interest (PI) identified in each image (I₀ to I₇) of the pyramid of Gaussian images;

orientable recursive filterings in each image (I₀ to I₇) of the pyramid of Gaussian images about each pixel of interest (PI), in order to calculate a bounded series expansion whose terms consist of the partial derivatives, from an order 0 to an order n, of the convolution function of the Gaussian function relating to said image, with the intensity of the relevant pixel of interest (PI);

generation of said first vector of characteristics relating to each pixel of interest (PI) by estimating each characteristic (E₁ to E_k) as one of the terms of the series expansions calculated for the images of the pyramid of Gaussian images.

7. Method according to claim 6, comprising a step of:

generation of said first vector of characteristics, with the addition, to the characteristics already calculated, of special characteristics, including the modulus of the gradient of the pixels of interest, the direction in which this gradient is a maximum, and the curvature of the lines of equal grey level.

8. Method of processing images in order automatically to detect points situated on the contour of an object (LV), referred to as key pixels (K₂, K₃, K₄), in an image referred to as the initial image (I₀), this method comprising a first phase including steps of:

storage in the initial image (I₀) in the form of a two-dimensional matrix of pixels of the intensity values of each pixel (A₀) labelled by its coordinates (x, y);

storage in the initial image (I₀) of data of regions of the object (LV) which are referred to as classes (C₁ to C_m) including classes which are referred to as corresponding classes (C₂, C₃, C₄) respectively containing;

the key pixels (K₂, K₃, K₄) to be detected;

selection of pixels of the initial image (I₀) which are referred to as pixels of interest (PI), on the contour of and inside and outside the object (LV);

generation of a first vector of characteristics (E₁ to E_k) for each of the pixels of interest (PI); and

classification of the pixels of interest (PI) into the said classes (C₁ to C_m) of the object (LV) on the basis of their respective vector of characteristics (E₁ to E_k); and

furthermore comprising a second phase including a step of selection of each of the key pixels (K₂, K₃, K₄) from the pixels of interest (PI) of the corresponding class (C₂, C₃, C₄).

9. Method according to claim 8 comprising, in the second phase, steps of:

generation of a second vector of characteristics (E'"'"'₁ to E'"'"'₁) relating to the pixels of interest, these characteristics coding the probabilities of presence of certain of said classes (C₁ to C_m) around the pixels of interest (PI);

classification of the pixels of interest (PI) into corresponding classes (C₂, C₃, C₄) and a class of the other pixels.

10. Method according to claim 9 comprising a step of:

selection of the key pixels (K₂, K₃, K₄) as those which fulfil a first condition consisting in the highest probability of belonging respectively to the corresponding class (C₁, C₂, C₃).

11. Method according to claim 10, in which the image to be processed is a cardiographic image where the object is the left ventricle (LV), and where the key pixels to be detected are the apex (K₂) and the two ends (K₃, K₄) of the aortic valve, this method comprising in the second phase a step of:

selection of the key pixels (K₂, K₃, K₄) as those which also fulfil a second geometrical condition according to which the ends of the aortic valve (K₃, K₄) form the base of an isosceles triangle of which the apex (K₂) is the vertex.

12. Device for processing images in order automatically to detect points situated on the contour of an object (LV), referred to as key pixels (K₂, K₃, K₄) in an image referred to as the initial image (I₀), this device comprising:

first storage means (MEM1) for storing, in the initial image (I₀) in the form of a two-dimensional matrix of pixels, the intensity values of each pixel (A₀) labelled by its coordinates (x, y), and data of regions of the object (LV), which are referred to as classes (C₁ to C_m), including classes referred to as corresponding classes (C₂, C₃, C₄) respectively containing the key pixels (K₂, K₃, K₄) to be detected;

means of selection of pixels of the initial image (I₀), referred to as pixels of interest (PI), on the contour of and inside and outside the object (LV);

first means of calculation (CP1), for generating a first vector of characteristics (E₁ to E_k) for each of the pixels of interest (PI);

a first neural network (NN1) for receiving at its inputs the first vector of characteristics and for performing a classification of the pixels of interest (PI) into said classes (C₁ to C_M) of the object (LV) on the basis of their respective vector of characteristics (E₁ to E_k).

13. Device according to claim 12 comprising second storage means (MEM2) and display means (DISPL) for respectively storing and displaying data formed by intensities of pixels labelled by their coordinates (x, y) in images in the form of a two-dimensional matrix of pixels.

14. Device for processing images according to claim 12 moreover comprising:

means of selection of each of the key pixels (K₂, K₃, K₄) from the pixels of interest (PI) of the corresponding class (C₂, C₃, C₄).

15. Device according to claim 14, comprising:

second means of calculation (CP2) for generating a second vector of characteristics (E'"'"'₁ to E'"'"'₁) relating to the pixels of interest;

a second neural network (NN2) for receiving at its inputs the second vector of characteristics and for classifying the pixels of interest (PI) into corresponding classes (C₂, C₃, C₄) and a class of the other pixels.

16. Device according to claim 15, moreover comprising:

means (F'"'"') for providing local image-data around each pixel of interest (PI) and drawn from images retorted to as probability images consisting of pixels of interest in said classes (C₁ to C₁₀) provided by the first neural network (NN1),

means of calculating the intensity of each pixel identified in each probability image,

recursive filters orientable about each pixel of interest (PI) for calculating in each probability image partial derivatives from an order 0 to an order p, in which device;

the second means of calculation (CP2) estimate each characteristic (E'"'"'₁ to E'"'"'₁) of the second vector of characteristics as those applied at each pixel of interest which code the probability of presence of certain of said classes (C₁ to C_m) around the pixels of interest (PI),

and the means of selection of the key pixels (K₂, K₃, K₄) retain the pixels which fulfil a condition consisting in the highest probability of belonging respectively to the corresponding class (C₁, C₂, C₃).

17. Device according to claim 16, for processing a cardiographic image where the object is the left ventricle (LV), where the key pixels to be detected are the apex (K₂) and the two ends (K₃, K₄) of the aortic valve, in which device:

the means of selection of the key pixels (K₂, K₃, K₄) retain those which also fulfil a geometrical condition according to which the ends of the aortic valve (K₃, K₄) form the base of an isosceles triangle of which the apex (K₂) is the vertex.

18. Device according to claim 12, in which the means of selection of the pixels of interest (PI) include:

gradient filters applied to the initial image (I₀) in order to select pixels having a gradient of maximum intensity which are situated on the contour of the object (LV) and to eliminate the other pixels;

means of calculation for sampling the initial image (I₀) at a lower frequency, preferentially preserving the pixels having a gradient of maximum intensity.

19. Device according to claim 18 moreover comprising:

means (F) for providing local and global image-data drawn from the initial image (I₀) around each pixel of interest (PI) including Gaussian function kernel multiresolution type filters applied to the initial image (I₀) and means of calculation for sub-sampling the filtered images to form a pyramid of Gaussian images (I₁ to I₇);

in which device;

the first means of calculation (CP1) generate said first vector of characteristics relating to each pixel of interest (PD on the basis of the image data of this pyramid of Gaussian images.

20. Device according to claim 19, in which:

the means (F) for providing local and global image-data include means for calculating a log-polar transformation function applied to the pixels of interest (PI) according to a chart formed of circular sectors;

and the first means of calculation (CP1) estimate each characteristic (E₁ to E_k) of said first vector of characteristics, as a mean of the intensities of the pixels in each circular sector of the chart relating to the log-polar transformation, by interpolation of the intensity of four neighboring pixels selected from one of the images of the pyramid of Gaussian images, which coincide with the four corner pixels delimiting said relevant circular sector, while taking account of the distance apart of these pixels, due to the frequency of sampling of the Gaussian image used.

21. Device according to claim 19, in which:

the means of calculating the intensity of each pixel of interest (PI) identified in each image (I₀ to I₇) of the pyramid of Gaussian images,

and recursive filters orientable about each pixel of interest (PI), for calculating, in each image (I₀ to I₇) of the pyramid of Gaussian images, a series expansion whose terms consist of the partial derivatives, from an order 0 to an order n, of the convolution function of the Gaussian function relating to said image, with the intensity of the relevant pixel of interest (PI), and

the first means of calculation (CP1) generate said first vector of characteristics relating to each pixel of interest (PI) by estimating each characteristic (E₁ to E_k) as one of the terms of the series expansions calculated for the images of the pyramid of Gaussian images.

22. Device according to claim 20, in which:

the first means of calculation (CP1) for generating said vector of characteristics comprise means for adding, to the characteristics already calculated, special characteristics, including the modulus of the gradient of the pixels of interest, the direction in which this gradient is a maximum, and the curvature of the lines of equal grey level.