Image recognition method and apparatus utilizing edge detection based on magnitudes of color vectors expressing color attributes of respective pixels of color image

• US 6,665,439 B1
• Filed: 04/07/2000
• Issued: 12/16/2003
• Est. Priority Date: 04/07/1999
• Status: Expired due to Term

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Takahashi, Eiji
• Primary Examiner(s): Mariam, Daniel G.
• Application Number:
  US09/545,580
• Time in Patent Office: 1,348 Days
• Field of Search: 382/151, 382/162, 382/164, 382/165, 382/181, 382/190, 382/197, 382/199, 382/203, 382/209, 382/216-221, 382/243, 382/254, 382/266, 382/269, 382/282, 382/291
• US Class Current: 382/199
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30184 Infrastructure
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

An image recognition apparatus operates on data of a color image to obtain an edge image expressing the shapes of objects appearing in the color image, the apparatus including a section for expressing the color attributes of each pixel of the image as a color vector, in the form of a set of coordinates of an orthogonal color space, a section for applying predetermined arrays of numeric values as edge templates to derive for each pixel a number of edge vectors each corresponding to a specific edge direction, with each edge vector obtained as the difference between weighted vector sums of respective sets of color vectors of two sets of pixels which are disposed symmetrically opposing with respect to the corresponding edge direction, and a section for obtaining the maximum modulus of these edge vectors as a value of edge strength for the pixel which is being processed. By comparing the edge strength of a pixel with those of immediately adjacent pixels and with a predetermined threshold value, a decision can be reliably made for each pixel as to whether it is actually located on an edge and, if so, the direction of that edge.

Claims

1. An image recognition method of processing image data of a color image which is represented as respective sets of color attribute data of an array of pixels, to successively operate on each of said pixels as an object pixel in order to determine whether said object pixel is located on an edge within said color image, and thereby derive shape data expressing an edge image corresponding to said color image, the method comprising steps of:

expressing said sets of color attribute data of each of said,pixels as respective color vectors, with each said color vector defined by a plurality of scalar values which are coordinates of an orthogonal color space;

for each of a plurality of predetermined edge directions, generating corresponding edge templates as an array of respectively predetermined numeric values;

extracting an array of color vectors as respective color vectors of a sub-array of said pixels, said sub-array of pixels being centered on said object pixel;

successively applying each of said edge templates to said array of color vectors in a predetermined array processing operation, to derive edge vectors respectively corresponding to said edge directions;

comparing the respective moduli of said derived edge vectors to obtain a value of edge strength for said object pixel, as a maximum value of moduilus of said edge vectors, and obtaining a possible edge direction for said object pixel as a direction corresponding to an edge vector having said maximum value of modulus; and

judging whether said object pixel is located on an actual ledge which is oriented in said possible edge direction, based upon comparing said edge strength of said object pixel with respective values of edge strength derived for pixels disposed adjacent to said object pixel.

2. The image recognition method according to claim 1, wherein said step of judging whether said object pixel is located on an actual edge which is oriented in said possible edge direction comprises comparing said edge strength of said object pixel with a predetermined threshold value and with respective values of edge strength of first and second adjacent pixels, said first and second adjacent pixels being located immediately adjacent to said object pixel and on opposing sides of said object pixel with respect to said possible edge direction, and judging that said object pixel is located on an actual edge which is oriented in said possible edge direction when it is found that said edge strength of said object pixel exceeds said threshold value and also exceeds said respective values of edge strength of said first and second adjacent pixels.

3. The image recognition method according to claim 1, wherein said numeric values constituting each of said edge templates include positive and negative values which are respectively disposed symmetrically opposite in relation to said corresponding edge direction within said edge template, and wherein said step of applying an edge template comprises performing an array multiplication operation between said edge template and said array of color vectors, and obtaining the vector sum of a result of said array multiplication operation as an edge vector.

4. The image recognition method according to claim 1, wherein said step of comparing the moduli of said derived edge vectors to obtain said value of edge strength of said object pixel comprises:

based on results of said comparison, selectively determining that said moduli have a first relationship whereby there is only a single maximum one of said modului, a second relationship whereby all of said moduli have an identical value, or a third relationship whereby a plurality of said moduli are greater than remaining one (s) of said moduli;

when said first relationship is determined, registering said maximum modulus as said value of edge strength of said object pixel, and registering information specifying a direction corresponding to the edge vector having said maximum modulus as the possible edge direction of said object pixel;

when said second relationship is determined, registering said identical value of modulus as said value of edge strength of said object pixel; and

when said third relationship is determined, arbitrarily selecting an edge vector having said greater value of modulus, registering said modulus value as said value of edge strength of said object pixel, and registering information for specifying a direction which corresponds to said selected edge vector as the possible edge direction of said object pixel.

5. The image recognition method according to claim 1, wherein said step of comparing the moduli of said derived edge vectors to obtain said value of edge strength of said object pixel comprises:

based on results of said comparison, selectively determining that said moduli have;

a first relationship whereby there is only a single maximum one of said moduli,

a second relationship whereby all of said moduli have an identical value, or

a third relationship whereby a plurality of said moduli are greater than remaining one(s) of said moduli;

when said first relationship is determined, registering said maximum modulus as said value of edge strength of said object pixel, and registering information specifying a direction corresponding to the edge vector having said maximum modulus, as a single candidate edge direction of said object pixel;

when said second relationship is determined, registering said identical value of modulus as said value of edge strength of said object pixel; and

when said third relationship is determined, registering said greater value of modulus as said value of edge strength of said object pixel, and registering information specifying each of respective directions corresponding to each of said plurality of edge vectors having said greater value of modulus, as respective candidate edge directions of said object pixel; and

wherein said step of judging whether said object pixel is located on an actual edge is performed by successively utilizing each of said candidate edge directions, until an actual edge is detected or all of said candidate edge directions have been utilized.

6. The image recognition method according to claim 1, wherein said step of expressing said sets of color attribute data as respective color vectors comprises performing a transform processing operation on each of said sets of color attribute data to derive a corresponding plurality of scalar values which constitute a set of coordinates of a predetermined color space.

7. The image recognition method according to claim 6, wherein said predetermined color space is an HSI (hue, saturation, intensity) color space.

8. The image recognition method according to claim 7, wherein said coordinates of said HSI color space are obtained in the form of polar coordinates, and further comprising a step of converting each said set of polar coordinates to a corresponding plurality of scalar values which are linear coordinates of an orthogonal color space.

9. The image recognition method according to claim 8, wherein said set of linear coordinates obtained corresponding to each of said pixels is derived such that an intensity value for said pixel is expressed by a specific one of said set of coordinates while hue and saturation values for said pixel are expressed by other ones of said set of coordinates, and further comprising a step of multiplying at least one of said coordinates of said set by an arbitrarily determined parameter value such as to alter a relationship between respective magnitudes of said intensity value and said hue and saturation values.

10. The image recognition method according to claim 7, further comprising a step of converting each of said sets of coordinates of said pixels for said HSI color space to a corresponding set of coordinates of a modified HSI color space, such that saturation values expressed in said modified HSI color space are modified in accordance with corresponding intensity values.

11. The image recognition method according to claim 10, wherein said saturation values in the modified HSI color space are decreased in accordance with decreases in corresponding intensity values, in relation to saturation values in said HSI color space.

12. The image recognition method according to claim 10, wherein said saturation values in the modified HSI color space are decreased in relation to saturation values in said HSI color space, in accordance with increases in corresponding intensity values from a predetermined median intensity value, and are moreover decreased in relation to saturation values in said HSI color space in accordance with decreases in corresponding intensity values from said predetermined median intensity value.

13. The image recognition method according to claim 10, wherein said step of converting each of said sets of coordinates of said pixels for said HSI color space to a corresponding set of coordinates of the modified HSI color space comprises applying a predetermined modification function to each of respective saturation values of said HSI color space to obtain modified saturation values.

14. The image recognition method according to claim 13, wherein said modification function is derived beforehand based upon a relationship between the intensity values and corresponding saturation values which are obtained by a transform into an HSI space having a specific size, with each of respective hue, saturation and intensity values expressed as a specific number of data bits.

15. A method of deriving for a selected pixel of a color image that is part of an array of pixels, for each of a plurality of predetermined edge directions, an edge strength value that corresponds to a specific one of a plurality of predetermined edge directions and is indicative of a degree of probability that said selected pixel is located on an edge between regions of respectively different color within said image, with said edge being oriented in said specific edge direction, the method comprising a set of steps performed for each of said edge directions of:

expressing color attributes of each of said pixels of said color image as a plurality of scalar values representing a color vector within an orthogonal color space;

obtaining a first weighted vector sum of a first set of pixels which are located adjacent to said selected pixel on one side thereof with respect to said specific edge direction and a second weighted vector sum of a second set of pixels which are located adjacent to said selected pixel on an opposite side from said first set with respect to said specific edge direction, and deriving the vector difference between said first and second weighted vector sums; and

obtaining a modulus of said vector difference,

and a step of judging the respective moduli thereby obtained respectively corresponding to said predetermined edge directions, to obtain said edge strength value as a largest one of said moduli.

16. An image recognition apparatus for processing image data of a color image which is represented as respective sets of color attribute data of an array of pixels, to successively operate on each of said pixels as an object pixel in order to determine whether said object pixel is located on an edge within said color image, and thereby derive shape data-expressing an edge image corresponding to said color image, the apparatus comprising:

color vector generating means for expressing said sets ofcolor attribute data of each of said pixels as respective color vectors, with each said color vector in the form of an array of a plurality of scalar values which are coordinates of an orthogonal color space;

edge template application means for generating a plurality of edge templates each formed of an array of respectively predetermined numeric values, with said edge templates corresponding to respective ones of a plurality of predetermined edge directions, thereby extracting an array of color vectors as respective color vectors of a sub-array of said pixels, with said sub-array of pixels being centered on said object pixel, and successively applying each of said edge templates to said array of color vectors in a predetermined array 5processing operation, to derive edge vectors respectively corresponding to said edge directions;

edge pixel determining means for comparing the respective moduli of said derived edge vectors to obtain a value of edge strength for said object pixel, as a maximum value of modulus of said edge vectors, thereby obtaining a possible edge direction for said object pixel as a direction corresponding to an edge vector having said maximum;

value of modulus, and for judging whether said object pixel is located on an actual edge that is oriented in said possible edge direction, based upon comparing said edge strength of said object pixel with respective values of edge strength derived for pixels disposed adjacent to said object pixel.

17. The image recognition apparatus according to claim 16, wherein said operation of judging whether said object pixel is located on an actual edge which is oriented in said possible edge direction comprises comparing said edge strength of said object pixel with a predetermined threshold value and with respective values of edge strength of of first and second adjacent pixels, said first and second adjacent pixels being located immediately adjacent to said object pixel and on opposing sides of said object pixel with respect to said possible edge direction, and judging that said object pixel is located on an actual edge which is oriented in said possible edge direction when it is found that said edge strength of said object pixel exceeds said threshold value and also exceeds said respective values of edge strength of said first and second adjacent pixels.

18. The image recognition apparatus according to claim 16, wherein said numeric values constituting each of said edge templates include positive and negative values which are respectively disposed symmetrically opposite in relation to said corresponding edge direction within said edge template, and wherein said operation of applying an edge template is executed by performing an array multiplication operation between said edge template and said array of color vectors, and obtaining the vector sum of a result of said array multiplication operation as an edge vector.

19. The image recognition apparatus according to claim 16, wherein said operation of comparing the moduli of said derived edge vectors to obtain said value of edge strength of said object pixel comprises:

based on results of said comparison, selectively determining that said moduli have;

a first relationship whereby there is only a single maximum one of said moduli,

a second relationship whereby all of said moduli have an identical value, or

a third relationship whereby a plurality of said moduli are greater than remaining one (s) of said moduli;

when said first relationship is determined, registering said maximum modulus as said value of edge strength of said object pixel, and registering information specifying a direction corresponding to the edge vector having said maximum modulus as the possible edge direction of said object pixel;

when said second relationship is determined, registering said identical value of modulus as said value of edge strength of said object pixel; and

when said third relationship is determined, arbitrarily selecting an edge vector having said greater value of modulus, registering said modulus value as said value of edge strength of said object pixel, and registering information that specifies that a direction corresponding to said selected edge vector is a possible edge direction of said object pixel.

20. The image recognition apparatus according to claim 16, wherein said operation of comparing the moduli of said derived edge vectors to obtain said value of edge strength of said object pixel comprises:

based on results of said comparison, selectively determining that said moduli have a first relationship whereby there is only a single maximum one of said moduli, a second relationship whereby all of said moduli have an identical value, or a third relationship whereby a plurality of said moduli are greater than remaining one (s) of said moduli;

when said first relationship is determined, registering said maximum modulus as said value of edge strength of said object pixel, and registering information specifying a direction corresponding to the edge vector having said maximum modulus, as a single candidate edge direction of said object pixel;

when said second relationship is determined, registering said identical value of modulus as said value of edge strength of said object pixel; and

when said third relationship is determined, registering said greater value of modulus as said value of edge strength of said object pixel, and registering information specifying each of respective directions corresponding to each of said plurality of edge vectors having said greater value of modulus, as respective candidate edge directions of said object pixel; and

wherein said operation of judging whether said object pixel is located on an actual edge is performed by successively utilizing each of said candidate edge directions, until an actual edge is detected or all of said candidate edge directions have been utilized.

21. The image recognition apparatus according to claim 16, wherein said operation of expressing said sets of color attribute data as respective color vectors is executed by performing a transform processing operation on each of said sets of color attribute data to derive a corresponding plurality of scalar values which constitute a set of, coordinates of a predetermined color space.

22. The image recognition apparatus according to claim 21, wherein said predetermined color space is an HIS (hue, saturation, intensity) color space.

23. The image recognition apparatus according to claim 22, wherein said coordinates of said HSI color space are obtained in the form of polar coordinates, and wherein said color vector generating means further comprises means for converting each said set of polar coordinates to a corresponding plurality of scalar values which are linear coordinates of an orthogonal color space.

24. The image recognition apparatus according to claim 23, wherein said set of linear coordinates obtained corresponding to each of said pixels is derived such that an intensity value for said pixel is expressed by a specific one of said set of coordinates while hue and saturation values for said pixel are expressed by other ones of said set of coordinates, and wherein said color vector generating means further comprises means for multiplying at least one of said coordinates of said set by an arbitrarily determined parameter value to thereby alter a relationship between respective magnitudes of said intensity value and said hue and saturation values.

25. The image recognition apparatus according to claim 22, wherein said color vector generating means further comprises means for converting each of said sets of coordinates of said pixels for said HSI color space to a corresponding set of coordinates of a modified HSI color space, such that saturation values expressed in said modified HSI color space are altered in accordance with corresponding intensity values.

26. The image recognition apparatus according to claim 25, wherein said saturation values in the modified HSI color space are decreased in accordance with decreases in corresponding intensity values, in relation to saturation values in said HSI color space.

27. The image recognition apparatus according to claim 25, wherein said saturation values in the modified HSI color space are decreased in relation to saturation values in said HSI color space, in accordance with increases in corresponding intensity values from a predetermined median value, and are moreover decreased in relation to saturation values in said HSI color space, in accordance with decreases in corresponding intensity values from said predetermined median value.

28. The image recognition apparatus according to claim 25, wherein said operation of converting each of said sets of coordinates of said pixels for said HSI color space to a corresponding set of coordinates of the modified HSI color space is executed by applying a predetermined modification function to each of respective saturation values of said HSI color space to obtain modified saturation values.

29. The image recognition apparatus according to claim 28, wherein said modification function is derived beforehand based upon a relationship between the intensity values and corresponding saturation values which are obtained by a transform into an HSI space having a specific size, with each of respective hue, saturation and intensity values expressed as a specific number of data bits.