Image recognition method and apparatus utilizing edge detection based on magnitudes of color vectors expressing color attributes of respective pixels of color image

• US 20040081355A1
• Filed: 10/15/2003
• Published: 04/29/2004
• Est. Priority Date: 04/07/1999
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Takahashi, Eiji
• Granted Patent: US 6,885,771 B2
• Time in Patent Office: Days
• US Class Current: 382/165
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30184 Infrastructure
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

An image recognition apparatus operates on data of a color image to obtain an edge image expressing the shapes of objects appearing in the color image, the apparatus including a section for expressing the color attributes of each pixel of the image as a color vector, in the form of a set of coordinates of an orthogonal color space, a section for applying predetermined arrays of numeric values as edge templates to derive for each pixel a number of edge vectors each corresponding to a specific edge direction, with each edge vector obtained as the difference between weighted vector sums of respective sets of color vectors of two sets of pixels which are disposed symmetrically opposing with respect to the corresponding edge direction, and a section for obtaining the maximum modulus of these edge vectors as a value of edge strength for the pixel which is being processed. By comparing the edge strength of a pixel with those of immediately adjacent pixels and with a predetermined threshold value, a decision can be reliably made for each pixel as to whether it is actually located on an edge and, if so, the direction of that edge.

Claims

1. An image recognition method of processing image data of a color image which is represented as respective sets of color attribute data of an array of pixels, to successively operate on each of said pixels as an object pixel for thereby determining whether said object pixel is located on an edge within said color image, and thereby derive shape data expressing an edge image corresponding to said color image, the method comprising steps of:

expressing said sets of color attribute data of each of said pixels as respective color vectors, with each said color vector defined by a plurality of scalar values which are coordinates of an orthogonal color space;

for each of a plurality of predetermined edge directions, generating a corresponding edge template as an array of respectively predetermined numeric values;

extracting an array of color vectors a respective color vectors of an array of said pixels, said array of pixels being centered on said object pixel;

successively applying each of said edge templates to said array of color vectors in a predetermined array processing operation, to derive edge vectors respectively corresponding to said edge directions;

comparing the respective moduli of said derived edge vectors to obtain a value of edge strength for said object pixel, as a maximum value of modulus of said edge vectors, and obtaining a possible edge direction for said object pixel as a direction corresponding to an edge vector having said maximum value of modulus; and

judging whether said object pixel is located on an actual edge which is oriented in said possible edge direction, based upon comparing said edge strength of said object pixel with respective values of edge strength derived for pixels disposed adjacent to said object pixel.

2. The image recognition method according to claim 1, wherein said step of judging whether said object pixel is located on an actual edge which is oriented in said possible edge direction comprises comparing said edge strength of said object pixel with a predetermined threshold value and with respective values of edge strength of first and second adjacent pixels, said first and second adjacent pixels being located immediately adjacent to said object pixel and on opposing sides of said object pixel with respect to said possible edge direction, and judging that said object pixel is located on an actual edge which is oriented in said possible edge direction when it is found that said edge strength of said object pixel exceeds said threshold value and also exceeds said respective values of edge strength of said first and second adjacent pixels.

3. The image recognition method according to claim 1, wherein said numeric values constituting each of said edge templates include positive and negative values which are respectively disposed symmetrically opposite in relation to said corresponding edge direction within said edge template, and wherein said step of applying an edge template comprises performing an array multiplication operation between said edge template and said array of color vectors, and obtaining the vector sum of a result of said array multiplication operation as an edge vector.

4. The image recognition method according to claim 1, wherein said step of comparing the moduli of said derived edge vectors to obtain said value of edge strength of said object pixel comprises:

based on results of said comparison, selectively determining that said moduli have a first relationship whereby there is only a single maximum one of said modului, a second relationship whereby all of said moduli have an identical value, or a third relationship whereby a plurality of said moduli are greater than remaining one(s) of said moduli;

when said first relationship is determined, registering said maximum modulus as said value of edge strength of said object pixel, and registering information specifying a direction corresponding to the edge vector having said maximum modulus as the possible edge direction of said object pixel;

when said second relationship is determined, registering said identical value of modulus as said value of edge strength of said object pixel; and

when said third relationship is determined, arbitrarily selecting an edge vector having said greater value of modulus, registering said modulus value as said value of edge strength of said object pixel, and registering information for specifying a direction which corresponds to said selected edge vector as the possible edge direction of said object pixel.

5. The image recognition method according to claim 1, wherein said step of comparing the moduli of said derived edge vectors to obtain said value of edge strength of said object pixel comprises:

based on results of said comparison, selectively determining that said moduli have a first relationship whereby there is only a single maximum one of said moduli, a second relationship whereby all of said moduli have an identical value, or a third relationship whereby a plurality of said moduli are greater than remaining one(s) of said moduli;

when said first relationship is determined, registering said maximum modulus as said value or edge strength of said object pixel, and registering information specifying a direction corresponding to the edge vector having said maximum modulus, as a single candidate edge direction of said object pixel;

when said second relationship is determined, registering said identical value of modulus as said value of edge strength of said object pixel; and

when said third relationship is determined, registering said greater value of modulus as said value of edge strength of said object pixel, and registering information specifying each of respective directions corresponding to each of said plurality of edge vectors having said greater value of modulus, as respective candidate edge directions of said object pixel;

and wherein said step of judging whether said object pixel is located on an actual edge is performed by successively utilizing each of said candidate edge directions, until an actual edge is detected or all of said candidate edge directions have been utilized.

6. The image recognition method according to claim 1, wherein said step of expressing said sets of color attribute data as respective color vectors comprises performing a transform processing operation on each of said sets or color attribute data to derive a corresponding plurality of scalar values which constitute a set of coordinates of a predetermined color space.

7. The image recognition method according to claim 6, wherein said predetermined color space is an HSI (hue, saturation, intensity) color space.

8. The image recognition method according to claim 7, wherein said coordinates of said HSI color space are obtained in the form of polar coordinates, and further comprising a step of converting each said set of polar coordinates to a corresponding plurality of scalar values which are linear coordinates of an orthogonal color space.

9. The image recognition method according to claim 8, wherein said set of linear coordinates obtained corresponding to each of said pixels is derived such that an intensity value for said pixel is expressed by a specific one of said set of coordinates while hue and saturation values for said pixel are expressed by other ones of said set of coordinates, and further comprising a step of multiplying at least one of said coordinates of said set by an arbitrarily determined parameter value such as to alter a relationship between respective magnitudes of said intensity value and said hue and saturation values.

10. The image recognition method according to claim 7, further comprising a step of converting each of said sets of coordinates of said pixels for said HSI color space to a corresponding set of coordinates of a modified HSI color space, such that saturation values expressed in said modified HSI color space are modified in accordance with corresponding intensity values.

11. The image recognition method according to claim 10, wherein said saturation values in the modified HSI color space are decreased in accordance with decreases in corresponding intensity values, in relation to saturation values in said HSI color space.

12. The image recognition method according to claim 10, wherein said saturation values in the modified HSI color space are decreased in relation to saturation values in said HSI color space, in accordance with increases in corresponding intensity values from a predetermined median intensity value, and are moreover decreased in relation to saturation values in said HSI color space in accordance with decreases in corresponding intensity values from said predetermined median intensity value.

13. The image recognition method according to claim 10, wherein said step of converting each of said sets of coordinates of said pixels for said HSI color space to a corresponding set of coordinates of the modified HSI color space comprises applying a predetermined modification function to each of respective saturation values of said HSI color space to obtain modified saturation values.

14. The image recognition method according to claim 13, wherein said modification function is derived beforehand based upon a relationship between the intensity values and corresponding saturation values which are obtained by a transform into an HSI space having a specific size, with each of respective hue, saturation and intensity values expressed as a specific number of data bits.

23. The image recording method according to claim 1, further comprising a step of converting said shape data expressing said edge image to shape data expressing a corresponding region image, and repetitive execution of a series of steps of:

determining all regions of said original region image which each have a size that is below a predetermined threshold value, as constituting a set of small regions which are each to be subjected to region combination;

selecting one of said set of small regions as a next small region which is to be subjected to said region combination;

for each of respective regions which are disposed immediately adjacent to said next small region, calculating a length of common boundary line with respect to said next small region, and determining one of said immediately adjacent regions which has a maximum value of said length of boundary line; and

combining said next small region with said adjacent region having the maximum length of common boundary line.

24. The image recording method according to claim 6, further comprising a step of converting said shape data expressing said edge image to shape data expressing a corresponding region image, and repetitive execution of a series of steps of:

determining all regions of said original region image which each have a size that is below a predetermined threshold value, as constituting a set of small regions which are each to be subjected to region combination;

selecting one of said set of small regions as a next small region which is to be subjected to said region combination;

for each of respective regions which are disposed immediately adjacent to said next small region, calculating a length of common boundary line with respect to said next small region, and determining one of said immediately adjacent regions which has a maximum value of said length of boundary line; and

combining said next small region with said adjacent region having the maximum length of common boundary line.

15. A method of deriving for a selected pixel of a color image which is formed of an array of pixels, for each of a plurality of predetermined edge directions, an edge strength value which corresponds to a specific one of a plurality of predetermined edge directions and is indicative of a degree of probability that said selected pixel is located on an edge between regions of respectively different color within said image, with said edge being oriented in said specific edge direction, the method comprising a set of steps performed for each of said edge directions of:

expressing the color attributes of each of said pixels of said color image as a plurality of scalar values representing a color vector within an orthogonal color space;

obtaining a first weighted vector sum of a first set of pixels which are located adjacent to said selected pixel on one side thereof with respect to said specific edge direction and a second weighted vector sum of a second set of pixels which are located adjacent to said selected pixel on an opposite side from said first set with respect to said specific edge direction, and deriving the vector difference between said first and second weighted vector sums; and

obtaining the modulus of said vector difference,

and a step of judging the respective moduli thereby obtained respectively corresponding to said predetermined edge directions, to obtain said edge strength value as the largest one of said moduli.

16. An image recognition method for operating on shape data expressing an original region image to obtain shape data expressing a region image in which specific small regions have been eliminated, comprising repetitive execution of a series of steps of:

selectively determining respective regions of said original region image as constituting a set of small regions which are each to be subjected to a region combining operation;

selecting one of said set of small regions as a next small region which is to be subjected to said region combining operation;

for each of respective regions which are disposed immediately adjacent to said next small region, calculating a length of common boundary line with respect to said next small region, and determining one of said immediately adjacent regions which has a maximum value of said length of boundary line; and

combining said next small region with said adjacent region having the maximum length of common boundary line.

17. The image recognition method according to claim 16, wherein said step of determining said set of small regions which are each to be subjected to a region combining operation is performed based upon judgement of respective size values of each of said regions of said color image.

18. The image recognition method according to claim 16, wherein said step of determining said set of small regions which are each to be subjected to a region combining operation is performed by selecting each of said regions of said color image having an area which is less than a predetermined threshold value.

19. The image recognition method according to claim 16, wherein said step of selecting one of said set of small regions as a next small region to be subjected to region combination is performed by selecting an arbitrary one of said set of small regions.

20. The image recognition method according to claim 16, wherein said step of selecting one of said set of small region as a next small region to be subjected to region combination is performed by selecting the smallest one of said set of small regions.

21. The image recognition method according to claim 16, wherein said step of selecting one of said set of small regions as a next small region to be subjected to region combination is based upon the respective sizes of said set of small regions.

22. The image recognition method according to claim 16, wherein said step of selecting one of said set of small regions as a next small region to be subjected to region combination is based upon the respective total sizes of sets of regions which are located immediately adjacent to respective ones of said set of small regions.

25. An image recognition apparatus for processing image data of a color image which is represented as respective sets of color attribute data of an array of pixels, to successively operate on each of said pixels as an object pixel for thereby determining whether said object pixel is located on an edge within said color image, and thereby derive shape data expressing an edge image corresponding to said color image, the apparatus comprising:

color vector generating means for expressing said sets of color attribute data of each of said pixels as respective color vectors, with each said color vector in the form of an array of a plurality of scalar values which are coordinates of an orthogonal color space;

edge template application means for generating a plurality of edge templates each formed of an array of respectively predetermined numeric values, with said edge templates corresponding to respective ones of a plurality of predetermined edge directions, for extracting an array of color vectors as respective color vectors of an array of said pixels, with said array of pixels centered on said object pixel, and successively applying each of said edge templates to said array of color vectors in a predetermined array processing operation, to derive edge vectors respectively corresponding to said edge directions;

edge pixel determining means for comparing the respective moduli of said derived edge vectors to obtain a value of edge strength for said object pixel, as a maximum value of modulus of said edge vectors, for obtaining a possible edge direction for said object pixel as a direction corresponding to an edge vector having said maximum value of modulus, and for judging whether said object pixel is located on an actual edge which is oriented in said possible edge direction, based upon comparing said edge strength of said object pixel with respective values of edge strength derived for pixels disposed adjacent to said object pixel.

26. The image recognition apparatus according to claim 25, wherein said operation of judging whether said object pixel is located on an actual edge which is oriented in said possible edge direction comprises comparing said edge strength of said object pixel with a predetermined threshold value and with respective values of edge strength of first and second adjacent pixels, said first and second adjacent pixels being located immediately adjacent to said object pixel and on opposing sides of said object pixel with respect to said possible edge direction, and judging that said object pixel is located on an actual edge which is oriented in said possible edge direction when it is found that said edge strength of said object pixel exceeds said threshold value and also exceeds said respective values of edge strength of said first and second adjacent pixels.

27. The image recognition apparatus according to claim 25, wherein said numeric values constituting each of said edge templates include positive and negative values which are respectively disposed symmetrically opposite in relation to said corresponding edge direction within said edge template, and wherein said operation of applying an edge template is executed by performing an array multiplication operation between said edge template and said array of color vectors, and obtaining the vector sum of a result of said array multiplication operation as an edge vector.

28. The image recognition apparatus according to claim 25, wherein said operation of comparing the moduli of said derived edge vectors to obtain said value of edge strength of said object pixel comprises:

based on results of said comparison, selectively determining that said moduli have a first relationship whereby there is only a single maximum one of said moduli, a second relationship whereby all of said moduli have an identical value, or a third relationship whereby a plurality of said moduli are greater than remaining one(s) of said moduli;

when said first relationship is determined, registering said maximum modulus as said value of edge strength of said object pixel, and registering information specifying a direction corresponding to the edge vector having said maximum modulus as the possible edge direction of said object pixel;

when said second relationship is determined, registering said identical value of modulus as said value of edge strength of said object pixel; and

when said third relationship is determined, arbitrarily selecting an edge vector having said greater value of modulus, registering said modulus value as said value of edge strength of said object pixel, and registering information which specifies that a direction corresponding to said selected edge vector is a possible edge direction of said object pixel.

29. The image recognition apparatus according to claim 25, wherein said operation of comparing the moduli of said derived edge vectors to obtain said value of edge strength of said object pixel comprises:

based on results of said comparison, selectively determining that said moduli have a first relationship whereby there is only a single maximum one of said moduli, a second relationship whereby all of said moduli have an identical value, or a third relationship whereby a plurality of said moduli are greater than remaining one(s) of said moduli;

when said first relationship is determined, registering said maximum modulus as said value of edge strength of said object pixel, and registering information specifying a direction corresponding to the edge vector having said maximum modulus, as a single candidate edge direction of said object pixel;

when said second relationship is determined, registering said identical value of modulus as said value of edge strength of said object pixel; and

when said third relationship is determined, registering said greater value of modulus as said value of edge strength of said object pixel, and registering information specifying each of respective directions corresponding to each of said plurality of edge vectors having said greater value of modulus, as respective candidate edge directions of said object pixel; and

wherein said operation of judging whether said object pixel is located on an actual edge is performed by successively utilizing each of said candidate edge directions, until an actual edge is detected or all of said candidate edge directions have been utilized.

30. The image recognition apparatus according to claim 25, wherein said operation of expressing said sets of color attribute data as respective color vectors is executed by performing a transform processing operation on each of said sets of color attribute data to derive a corresponding plurality of scalar values which constitute a set of coordinates of a predetermined color space.

31. The image recognition apparatus according to claim 30, wherein said predetermined color space is an HSI (hue, saturation, intensity) color space.

32. The image recognition apparatus according to claim 31, wherein said coordinates of said HSI color space are obtained in the form of polar coordinates, and wherein said color vector generating means further comprises means for converting each said set of polar coordinates to a corresponding plurality of scalar values which are linear coordinates of an orthogonal color space.

33. The image recognition apparatus according to claim 32, wherein said set of linear coordinates obtained corresponding to each of said pixels is derived such that an intensity value for said pixel is expressed by a specific one of said set of coordinates while hue and saturation values for said pixel are expressed by other ones of said set of coordinates, and wherein said color vector generating means further comprises means for multiplying at least one of said coordinates of said set by an arbitrarily determined parameter value to thereby alter a relationship between respective magnitudes of said intensity value and said hue and saturation values.

34. The image recognition apparatus according to claim 31, wherein said color vector generating means further comprises means for converting each of said sets of coordinates of said pixels for said HSI color space to a corresponding set of coordinates of a modified HSI color space, such that saturation values expressed in said modified HSI color space are altered in accordance with corresponding intensity values.

35. The image recognition apparatus according to claim 34, wherein said saturation values in the modified HSI color space are decreased in accordance with decreases in corresponding intensity values, in relation to saturation values in said HSI color space.

36. The image recognition apparatus according to claim 34, wherein said saturation values in the modified HSI color space are decreased in relation to saturation values in said HSI color space, in accordance with increases in corresponding intensity values from a predetermined median value, and are moreover decreased in relation to saturation values in said HSI color space, in accordance with decreases in corresponding intensity values from said predetermined median value.

37. The image recognition apparatus according to claim 34, wherein said operation of converting each of said sets of coordinates of said pixels for said HSI color space to a corresponding set of coordinates of the modified HSI color space is executed by applying a predetermined modification function to each of respective saturation values of said HSI color space to obtain modified saturation values.

38. The image recognition apparatus according to claim 37, wherein said modification function is derived beforehand based upon a relationship between the intensity values and corresponding saturation values which are obtained by a transform into an HSI space having a specific size, with each of respective hue, saturation and intensity values expressed as a specific number of data bits.

46. The image recording apparatus according to claim 25, further comprising:

means for converting said shape data expressing said edge image to shape data expressing a region image in which respective regions are separately identified;

small region detection means for selectively determining respective regions of said region image as constituting a set of small regions which are each to be subjected to region combination;

region combination determining means for selecting one of said set of small regions as a next small region which is to be subjected to said region combination; and

,

region combining means for calculating respective values of common boundary line between said next small region and each of the regions which are located immediately adjacent to said next small region, for determining one of said immediately adjacent regions which has a maximum value of said length of boundary line, and for combining said next small region with said immediately adjacent region having the maximum length of common boundary line.

47. The image recording apparatus according to claim 30, further comprising:

means for converting said shape data expressing said edge image to shape data expressing a region image in which respective regions are separately identified;

small region detection means for selectively determining respective regions of said region image as constituting a set of small regions which are each to be subjected to region combination;

region combination determining means for selecting one of said set of small regions as a next small region which is to be subjected to said region combination; and

,

region combining means for calculating respective values of common boundary line between said next small region and each of the regions which are located immediately adjacent to said next small region, for determining one of said immediately adjacent regions which has a maximum value of said length of boundary line, and for combining said next small region with said immediately adjacent region having the maximum length of common boundary line.

39. An image recognition apparatus for operating on shape data expressing an original region image to obtain shape data expressing a region image in which specific small regions have been eliminated, comprising:

small region detection means for selectively determining respective regions of said original region image as constituting a set of small regions which are each to be subjected to region combination;

region combination determining means for selecting one of said set of small regions as a next small region which is to be subjected to said region combination; and

,

region combining means for calculating respective values of common boundary line between said next small region and each of the regions which are located immediately adjacent to said next small region, for determining one of said immediately adjacent regions which has a maximum value of said length of boundary line, and for combining said next small region with said adjacent region having the maximum length of common boundary line.

40. The image recognition apparatus according to claim 39, wherein said small region detection means comprises means for determining said set of small regions which are each to be subjected to a region combining operation based upon judgement of respective size values of each of said regions of said color image.

41. The image recognition apparatus according to claim 39, wherein said region combination determining means comprises means for determining one of said set of small regions as said next small region to be subjected to region combination, by selecting an arbitrary one of said set of small regions.

42. The image recognition apparatus according to claim 39, wherein said region combination determining means determines one of said set of small regions, as said next small region to be subjected to region combination, by selecting the smallest one of said set of small regions.

43. The image recognition apparatus according to claim 39, wherein said region combination determining means determines one of said set of small regions, as said next small region to be subjected to region combination, based upon the respective sizes of said set of small regions.

44. The image recognition apparatus according to claim 39, wherein said region combination determining means determines one of said set of small regions, as said next small region to be subjected to region combination, based upon the respective total sizes of sets of regions which are located immediately adjacent to respective ones of said set of small regions.

45. The image recording apparatus according to claim 39, wherein said small region detection means comprises means for selecting respective ones of said regions of the original region image that are smaller than a predetermined threshold value as said small regions which are to be subjected to region.combination.