System and method for human hand motion detection by skin color prediction

• US 8,050,494 B2
• Filed: 05/23/2008
• Issued: 11/01/2011
• Est. Priority Date: 05/23/2008
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Guan, Haiying, Xu, Ning
• Primary Examiner(s): Sherali; Ishrat I
• Application Number:
  US12/126,688
• Publication Number:
  US 20090290762A1
• Time in Patent Office: 1,257 Days
• Field of Search: 382162-167, 382/103, 382/107, 382117-118, 382224-225, 382/168, 382/172, 382/173, 382/232, 348/169, 375/240.01
• US Class Current: 382/162
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

Systems and methods for motion detection of human skin within temporally adjacent electronic images are provided. Motion detection is accomplished by analyzing the color values of selected pixels within the electronic images represented in CbCr color space. Histogram distributions which represent skin colors and non-skin colors in CbCr color space are modeled in order to provide likelihoods that a selected color value appears within the histogram distributions. Posterior skin probability values, indicating the probability that a selected pixel having a given CbCr color value represents human skin, are calculated from these likelihoods. For each of the selected pixels, an intensity difference of the pixel between the electronic images is compared to an adaptive intensity threshold which is a function of the posterior skin probability in order to determine whether the pixel is in motion.

Claims

1. A method of detecting whether a selected pixel in an electronic image F(t), having a pixel color, represents human skin in motion, comprising:

calculating a first probability, P₁, indicating a likelihood that the color of the selected pixel appears within a first distribution of pixel colors observed in human skin, wherein the color of the selected pixel and the first distribution of pixel colors are represented in the CbCr color space;

calculating a second probability, P₂, indicating a likelihood that the selected pixel color appears within a second distribution of pixel colors that are observed outside of human skin, wherein the second distribution of pixel colors are represented in the CbCr color space; and

calculating a third probability, P_K, indicating a probability that the selected pixel represents human skin, wherein P_K is a function of P₁ and P₂;

wherein the selected pixel is determined to represent human skin in motion if an intensity difference, I, between the intensities of the selected pixel within F(t) and a temporally adjacent electronic image F(t−

Δ

t) is greater than a threshold intensity value, T, which is a function of P_K.

2. The method of claim 1, wherein the second probability P₂ is calculated from the pixel color of the selected pixel, x′

, using a single Gaussian model, P_NS, that models a histogram of the second distribution of pixel colors, wherein P_NS is described by a mean, μ

_NS, and a covariance, Σ

_NS, which are obtained by fitting P_NS to the histogram of the second distribution of pixel colors.

3. The method of claim 1, wherein the second probability P₂ is given by:

4. The method of claim 2, wherein the first probability P₁ is calculated from the pixel color of the selected pixel, x′

, using a Gaussian Mixture Model, P_S, that models a histogram of the first distribution of pixel colors, wherein P_S comprises of a sum of weighted Gaussian distributions, p_i, and wherein each of p_i are described by a mean, μ

_S,i, and covariance, Σ

_S,i obtained by fitting P_S to the histogram of the first distribution of pixel colors using an expectation maximization process.

5. The method of claim 3, wherein P₁ is given by:

6. The method of claim 4, wherein P_K is further a function of the ratio of the number of pixels within the first distribution of pixel colors and the total number of pixels in the first and second distributions of pixel colors, f₁, and the ratio of the number of pixels within the second distribution of pixel colors and the total number of pixels in the first and second distributions of pixel colors, f₂.

7. The method of claim 5, wherein P_K is calculated according to:

8. The method of claim 1, further comprising a filtering operation which identifies pixels of F(t) which are not to selected as human skin pixels in motion, the filtering operation comprising:

calculating a ratio of a red color value to a green color value, R/G, of the pixel color of the selected pixel represented in the RGB color space; and

setting P_K of the selected pixel to zero if R/G is greater than a selected value.

9. The method of claim 1, wherein the threshold intensity value, T, is given by the equation:

T=(T₁−

T₂)P_K+T₂

wherein T₁ and T₂ are selected intensity values and wherein T₂ is greater than T₁.

10. A system for identifying the motion of a selected pixel having a pixel color within a plurality of temporally adjacent electronic images, comprising:

a skin likelihood calculator,

wherein the skin likelihood calculator employs a skin color model which models a first histogram distribution of pixel colors representing human skin in order to calculate a skin likelihood, P₁, representing a likelihood of observing the color of the selected pixel in human skin;

and wherein the skin likelihood calculator further employs a non-skin color model which models a second histogram distribution of pixel colors which do not represent human skin in order to calculate a non-skin likelihood, P₂, representing a likelihood of observing the color of the selected pixel outside of human skin wherein the pixel color of the selected pixel and the first and second histogram distributions are represented in the CbCr color space;

a posterior probability calculator which employs a Bayesian equation, the color of the selected pixel, and the skin and non-skin likelihoods P₁ and P₂ in order to calculate a probability that the selected pixel represents human skin; and

a motion detection engine which determines that the selected pixel represents human skin in motion if a difference in intensity of the selected pixel between a first electronic image and a second electronic image selected from the plurality of electronic images is greater than a threshold intensity value, T, wherein T is a function of P_K.

11. The system of claim 10, wherein the threshold intensity value is a linear function of P_K.

12. The system of claim 11, wherein the threshold intensity value, T, is given by:

T=(T₁−

T₂)P_K+T₂

wherein T₁ and T₂ are selected intensity values and T₂ is greater than T₁.

13. The system of claim 10, wherein the non-skin color probability P₂ is calculated according to:

14. The system of claim 10, wherein the skin likelihood P₁ is calculated according to:

15. The system of claim 10, wherein the skin probability P_K is given by:

16. A method of detecting the motion of human skin within a video, comprising:

receiving at least a first video frame and a second video frame which are temporally adjacent to one another;

selecting a pixel from the first video frame, the pixel having a pixel color;

calculating a skin probability, P_K that the selected pixel represents human skin, wherein the skin probability P_K is a function of a likelihood of observing the selected pixel'"'"'s color in a first plurality of pixels representing human skin and a likelihood of observing the selected pixel'"'"'s color in a second plurality of pixels not representing human skin, wherein the pixel color of the selected pixel and the first and second plurality of pixels are represented in the CbCr color space;

comparing an intensity difference, I, between the selected pixel within the first and second video frames to a threshold intensity, T, comprising a function of P_K;

wherein the selected pixel is considered to represent the motion of human skin if the value of I is greater than T.

17. The method of claim 16, wherein the likelihood of observing the color of the selected pixel in the first plurality of pixels is determined from a first probability distribution, P_S, comprising a plurality of Gaussian distributions which are fit to a histogram distribution of the pixel colors of the first plurality of pixels using an expectation maximization process.

18. The method of claim 17, wherein P₁ is given by the output of the first probability distribution P_S for the pixel color of the selected pixel color, x′

, comprising;

19. The method of claim 17, wherein the likelihood of observing the color of the selected pixel in the second plurality of pixels is determined from a second probability distribution, P_NS, comprising a single Gaussian distribution which is fit to a histogram distribution of the pixel colors of the second plurality of pixels.

20. The method of claim 19, wherein P₂ is given by the output of the second probability distribution P_NS for the pixel color of the selected pixel color, x′

, comprising;

21. The method of claim 20, wherein the probability P_K comprises:

22. The method of claim 16, wherein the threshold intensity value, T, is given by:

T=(T₁−

T₂)P_K+T₂

wherein T₁ and T₂ are selected intensity values and wherein T₂ is greater than T₁.

23. A system for detecting human motion within a video, comprising:

a video source capable of transmitting a plurality of video frames; and

a motion evaluation system which receives at least a first and a second temporally adjacent video frames from the video source, the motion evaluation system comprising a skin probability calculator and a motion detection engine;

wherein the skin probability calculator calculates a posterior skin probability, P_K, for at least one selected pixel within the plurality of video frames represents human skin, the posterior skin probability P_K comprising a probability of observing the pixel color of the selected pixel in a first histogram distribution of pixel colors representing human skin, P_S, and a probability of observing the pixel color of the selected pixel in a second histogram distribution of pixel colors which do not represent human skin, P_NS,

wherein the pixel color of the selected pixel and the first and second histogram distributions are represented in the CbCr color space; and

wherein motion detection engine determines that the selected pixel is in motion if an intensity difference of the selected pixel between the first electronic image and the second electronic image is greater than a threshold intensity, T, which comprises a function of P_K.

24. The system of claim 23, wherein the likelihood of observing the pixel color of the selected pixel in the first histogram distribution is determined from a first probability distribution, P_S, comprising a plurality of Gaussian distributions which are fit to the first histogram distribution using an expectation maximization process.

25. The system of claim 23, wherein the likelihood of observing the pixel color of the selected pixel in the second histogram distribution is determined from a second probability distribution, P_NS, comprising a single Gaussian distribution which is fit to the second histogram distribution.