Automatic segmentation-based grass detection for real-time video

• US 6,832,000 B2
• Filed: 03/28/2001
• Issued: 12/14/2004
• Est. Priority Date: 03/28/2001
• Status: Expired due to Term

• Current Assignee: Funai Electric Co., Ltd.
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Herman, Stephen, Janssen, Johan, Bellers, Erwin, Wendorf, James
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US09/819,290
• Publication Number:
  US 20020140815A1
• Time in Patent Office: 1,357 Days
• Field of Search: 382/162, 382/164-165, 382/167, 382/170, 382/171, 382/173, 382/224, 382/228, 382/263, 382/266, 382/274
• US Class Current: 382/165
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics

Abstract

Pixels in a video image may be segmented based upon selected criteria, such as a common color, texture, shape, amplitude range or temporal variation. Color values for these pixels may be used to calculate a color probability function which indicates the probability that the color value of the pixel will lie within a designated range of values. The pixels are also used to calculate a texture probability function that indicates whether the pixel represents a designated texture. Pixels that are assigned to a given segment may then be further processed to improve the quality of an image. In this manner, pixels that identify grass, sky, human skin, etc. may be identified and processed in order to achieve a more pleasing appearance.

Claims

1. A method of segmenting pixels in an image based upon selected criteria, comprising:

obtaining a signal that represents color values for pixels in the image;

calculating a color probability function that estimates, for each pixel in the image, the probability that the color value of the pixel will lie within a range of values that represents a designated color;

calculating a texture probability function that estimates, for each pixel in the image, the probability that the pixel represent a designated texture;

classifying each pixel in the based upon a combination of its color probability function and its texture probability function; and

processing each pixel in a designated classification, wherein the processing is proportional to said combination of color probability function and texture probability function.

2. A method as claimed in claim 1 wherein said signal is a baseband video signal.

3. A method as claimed in claim 1 wherein said color probability function and said texture probability function are defined in a YUV color space.

4. A method as claimed in claim 3 wherein said color probability function is defined as P_color=exp(−

(((y−

y₀)/σ

y)²+((u−

u₀)/σ

u)²+((v−

v₀)/σ

v)²)) where y represents a pixel luminance value, and u and v represent color coordinates in a YUV color space.

5. A method as claimed in claim 4 wherein y₀ is equal to about 112, σ

y is equal to about 40, u₀ is equal to about 110, σ

u is equal to about 8, v₀ is equal to about 124, and σ

v is equal to about 7.

6. A method as claimed in claim 3 wherein said texture probability function is defined as P_texture=((t/sqrt(m*t))*exp(−

((t−

m)/s)²), where t represents a root-mean-square variation of a pixel value luminance in a window surrounding the pixel, m is a value on a luminance scale that describes the location of a peak of the function and s is a value on said luminance scale that describes a width of said luminance function.

7. A method as claimed in claim 6, wherein said window is a 5×

5 pixel square, m is equal to 4 on a luminance scale with a peak value of 255 and s is equal to 5.

8. A method as claimed in claim 6 further comprising obtaining the parameter t further comprises:

calculating a spatial derivative of a luminance signal; and

filtering said luminance signal spatial derivative.

9. A method as claimed in claim 8 wherein said spatial derivative calculating step further comprises calculating a difference between a two dimensional luminance signal for the pixel and a neighboring luminance signal, and calculating an absolute value of said difference.

10. A method as claimed in claim 6 wherein m₁ is equal to about 0.3 and σ

t is equal to about 15.

11. A method as claimed in claim 1 wherein combining said color probability function and said texture probability function further comprises calculating a product of said color probability function and said texture probability function.

12. A method as claimed in claim 1 wherein said processing comprises altering the color of all pixels with said designated classification.

13. A method as claimed in claim 1 wherein said processing comprises increasing the frequency of all pixels with said designated classification.

14. A method as claimed in claim 1 wherein said processing comprises increasing the brightness of all pixels with said designated classification.

15. A method as claimed in claim 1 wherein said processing comprises reducing the noise processing for all pixels with said designated classification.

16. A method as claimed in claim 1 wherein at least some of said pixels are classified as representing grass.

17. A method as claimed in claim 1 wherein said color probability function and said texture probability function are defined in a HSV color space.

18. A device for segmenting a received video image signal based upon selected criteria, wherein the video image signal directs illumination of pixels on a video display, comprising:

a receiver that accepts the video image signal;

a color circuit that generates an output for a plurality of pixels, based upon a probability that a pixel luminance value and a pixel color value lie within designated ranges, wherein said color probability is defined as P_color=exp(−

(((y−

y₀)/σ

y)²+((u−

u₀)/σ

u)²+((v−

v₀)/σ

v)²)) where y represents said pixel luminance value, and u and v represent color coordinates in a YUV color space;

a texture circuit that generates an output for said plurality of pixels, based upon a probability that said pixels represent at least a portion of an object that has a designated texture, wherein said texture probability is defined as P_texture=((t/sqrt(m*t))*exp(−

((t−

m)/s)²), where t represents a root-mean-square variation of a pixel value luminance in a window surrounding the pixel, m is a value on a luminance scale that describes the location of a peak of the function and s is a value on said luminance scale that describes a width of said luminance function; and

a pixel segmenting circuit which separates said pixels into at least two classes, based upon a product of said color circuit output and said texture circuit output.

19. A method of segmenting pixels in an image based upon selected criteria, comprising:

obtaining a baseband video signal;

using a function defined as P_color=exp(−

(((y−

y₀)/σ

y)²+((u−

u₀)/σ

u)²+((v−

v₀)/σ

v)²)) where y represents a pixel luminance value, and u and v represent color coordinates in a YUV color space to calculate a color probability function for pixels in said baseband video signal, wherein said color probability function estimates, for each pixel in the image, the probability that the color value of the pixel will lie within a range of values that represents a designated color;

using a function defined as P_texture=((t/sqrt(m*t))*exp(−

((t−

m)/s)²), where t represents a root-mean-square variation of a pixel value luminance in a window surrounding the pixel, m is a value on a luminance scale that describes the location of a peak of the function and s is a value on said luminance scale that describes a width of said luminance function to calculate a texture probability for said baseband video signal, wherein said texture probability function estimates, for each pixel in the image, the probability that the pixel represent a designated texture; and

classifying each pixel in the image based upon a product of its color probability function and its texture probability function.

20. A method of segmenting pixels in an image based upon selected criteria, comprising:

obtaining a signal that represents color values for pixels in the image;

calculating a color probability function that estimates, for each pixel in the image, the probability that the color value of the pixel will lie within a range of values that represents the color green;

calculating a texture probability function that estimates, for each pixel in the image, the probability that the pixel represent a designated texture;

classifying at least some of said pixels as representing grass based upon a combination of their color probability function and their texture probability function; and

processing said pixels classified as representing grass, wherein the processing is proportional to said combination of color probability function and texture probability function.

21. A device for segmenting a received video image signal based upon selected criteria, wherein the video image signal directs illumination of pixels on a video display, comprising:

a receiver that accepts the video image signal;

a color circuit that generates a signal that indicates a probability that a pixel luminance value and a pixel color value lie within designated ranges;

a texture circuit that generates a signal that indicates a probability that a pixel represents at least a portion of an object that has a designated texture;

a pixel segmenting circuit that separates said pixels into at least two classes based upon a combination of said color circuit signal and said texture circuit signal; and

a processing circuit that enhances pixels in a one of said at least two classes, wherein the enhancement is proportional to said combination of said color circuit signal and said texture circuit signal.

22. A device as claimed in claim 21 wherein said color circuit signal is defined by the function P_color=exp(−

(((y−

y₀)/σ

y)²+((u−

u₀)/σ

u)²+((v−

v₀)/σ

v)²)) where y represents said pixel luminance value, and u and v represent color coordinates in a YUV color space.

23. A device as claimed in claim 21 wherein said texture circuit signal is defined by the function P_texture=((t/sqrt(m*t))*exp(−

((t−

m)/s)²), where t represents a root-mean-square variation of a pixel value luminance in a window surrounding the pixel, m is a value on a luminance scale that describes the location of a peak of the function and s is a value on said luminance scale that describes a width of said luminance function.