Multiple person detection apparatus and method

• US 20050152582A1
• Filed: 11/29/2004
• Published: 07/14/2005
• Est. Priority Date: 11/28/2003
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Yoon, Sengmin, Kee, Seokcheol
• Application Number:
  US10/998,150
• Publication Number:
  US 20050152582A1
• Time in Patent Office: Days
• US Class Current: 382/115
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

A multiple person detection apparatus and method. The multiple person detection apparatus includes a skin color detection unit, which detects at least one skin color region from a picked-up frame image by using skin color information, a candidate region determination unit, which determines whether or not each of the skin color regions belongs to a person candidate region, and a person determination unit, which determines whether or not the skin color region belonging to the person candidate region corresponds to a person by using person shape information.

Claims

1. A multiple person detection apparatus comprising:

a skin color detection unit, which detects at least one skin color region from a picked-up frame image by using skin color information;

a candidate region determination unit, which determines whether each of the skin color regions belongs to a person candidate region; and

a person determination unit, which determines whether the skin color region belonging to the person candidate region corresponds to a person by using person shape information.

2. The multiple person detection apparatus according to claim 1, wherein the skin color detection unit comprises:

a color normalization unit, which normalizes colors of pixels of the frame image;

a modeling unit, which performs a Gaussian modeling process on the normalized frame image to highlight pixels having colors similar to skin color; and

a labeling unit, which performs a labeling process on pixels having pixel values above a predetermined threshold value among the pixels having colors similar to the highlighted skin color to detect at least one skin color region, and generates sizes and weight centers of the skin color regions.

3. The multiple person detection apparatus according to claim 1, wherein the candidate determination unit normalizes the skin color regions detected by the skin color detection unit with a predetermined size, and determines whether each of the normalized skin color regions belongs to the person candidate region by using a Mahalanobis distance map.

4. The multiple person detection apparatus according to claim 1, wherein the person determination unit comprises:

an edge image generation unit, which generates an edge image for the person candidate region;

a model image storage unit, which stores an edge image of a model image;

a similarity evaluation unit, which evaluates similarity between the edge image of the model image and the edge image generated by the edge image generation unit; and

a determination unit, which determines whether the person candidate region corresponds to a person based on the evaluated similarity.

5. The multiple person detection apparatus according to claim 4, wherein the model image is constructed with at least one of a front model image, a left model image, and a right model image.

6. A multiple person detection method comprising:

detecting at least one skin color region from a picked-up frame image by using skin color information;

determining whether each of the skin color regions belongs to a person candidate region; and

determining whether the skin color region belonging to the person candidate region corresponds to a person by using person shape information.

7. The multiple person detection method according to claim 6, wherein the detecting at least one skin color region comprises:

normalizing colors of pixels of the frame image;

performing a Gaussian modeling process on the normalized frame image to highlight pixels having colors similar to skin color; and

performing a labeling process on pixels having pixel values above a predetermined threshold value among the pixels having colors similar to the highlighted skin color to detect at least one skin color region, and generating sizes and centers of weight of the skin color regions.

8. The multiple person detection method according to claim 7, wherein the detecting at least one skin color region further comprises, prior to detecting at least one skin color region, smoothing an RGB histogram of the frame image by equalizing the frame image.

9. The multiple person detection method according to claim 7, wherein, in normalizing colors of pixels of the frame image, the colors are normalized in accordance with the following equation:

$r=\frac{R}{R+G+B},g=\frac{G}{R+G+B},b=\frac{B}{R+G+B}$ $r+g+b=1,$

wherein r, g, and b denote normalized color signals, and R, G, and B denote color signals of the input frame image.

10. The multiple person detection method according to claim 7, wherein, in performing a Gaussian modeling process, the Gaussian modeling process is performed in accordance with the following equation:

$\begin{array}{c}Z<br><br>\left(x,y\right)=<br><br>G<br><br>\left(r<br><br>\left(x,y\right),g<br><br>\left(x,y\right)\right)\\ =<br><br>\frac{1}{2<br><br>{\mathrm{\pi <br><br>\sigma <br><br>}}_r<br><br>{\mathrm{\sigma <br><br>}}_g}<br><br>\exp <br><br>\left[-\frac{1}{2}<br><br>\left({\left(\frac{r<br><br>\left(x,y\right)-m_r}{{\mathrm{\sigma <br><br>}}_r}\right)}^2+{\left(\frac{g<br><br>\left(x,y\right)-m_g}{{\mathrm{\sigma <br><br>}}_g}\right)}^2\right\}\right],\end{array}$

wherein m_r and mg denote averages of colors r and g of multiple skin color models and σ

_r and σ

_g denote standard deviations of colors r and g of the multiple skin color models.

11. The multiple person detection method according to claim 6, wherein the determining whether or not each of the skin color regions belongs to a person candidate region comprises:

normalizing the detected skin color regions with a predetermined size; and

determining whether each of the normalized skin color regions belongs to the person candidate region.

12. The multiple person detection method according to claim 11, wherein the determining whether each of the normalized skin color regions belongs to the person candidate region is performed by using a Mahalanobis distance map.

13. The multiple person detection method according to claim 12, wherein, the Mahalanobis distance map is obtained by:

partitioning the normalized image into M (horizontal)×

N (vertical) blocks;

obtaining an average of distances of blocks using the following equation;

${\stackrel{\_}{x}}_l=\frac{1}{\mathrm{pq}}<br><br>\underset{\left(x,t\right)\in <br><br>X_1}{\overset{\text{\hspace{1em}<br><br>}}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>x_{s,t}$

wherein p and q denote pixel numbers in the horizontal and vertical directions of each block, respectively, X denotes total blocks, and x denotes a pixel value in each block;

obtaining the deviation of pixel values of each block using the following equation;

$\underset{l}{\sum <br><br>}<br><br>=\frac{1}{\mathrm{pq}}<br><br>\underset{x\in <br><br>X_l}{\overset{\text{\hspace{1em}<br><br>}}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(x-{\stackrel{\_}{x}}_l\right)<br><br>{\left(x-{\stackrel{\_}{x}}_l\right)}^T$

obtaining the Mahalanobis distance d(i, j) of each of the blocks and the Mahalanobis distance map D having the form of a matrix (M×

N)×

(M×

n) by using the following equations;

$d_{\left(i,j\right)}={\left({\stackrel{\_}{x}}_i,{\stackrel{\_}{x}}_j\right)}^{\mathrm{\prime <br><br>}}<br><br>{\left(\underset{i}{\sum <br><br>}<br><br>+\underset{j}{\sum <br><br>}\right)}^{-1}<br><br>\left({\stackrel{\_}{x}}_i-{\stackrel{\_}{x}}_j\right)<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{and}$$D=\left[\begin{array}{cccc}0& d_{\left(1,2\right)}& \mathrm{\dots <br><br>}& d_{\left(1,\mathrm{MN}\right)}\\ d_{\left(2,1\right)}& 0& \mathrm{\dots <br><br>}& d_{\left(2,\mathrm{MN}\right)}\\ \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}\\ d_{\left(\mathrm{MN},1\right)}& d_{\left(\mathrm{MN},2\right)}& \mathrm{\dots <br><br>}& 0\end{array}\right]$

14. The multiple person detection method according to claim 6, wherein the determining whether the skin color region belonging to the person candidate region corresponds to a person comprises:

generating an edge image for the person candidate region;

evaluating similarity between an edge image of a model image and the generated edge image;

determining based on the evaluated similarity whether the person candidate region corresponds to a person.

15. The multiple person detection method according to claim 14, wherein the similarity is evaluated based on a Hausdorff distance.

16. The multiple person detection method according to claim 15, wherein the input edge image A has m edges, and the model image B has n edges,

wherein the Hausdorff distance is obtained by using the following equations:

$H<br><br>\left(A,B\right)=\max <br><br>\left(h<br><br>\left(A,B\right),h<br><br>\left(B,A\right)\right)<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{and}$$h<br><br>\left(A,B\right)=\underset{a\in <br><br>A}{\max }<br><br>\underset{b\in <br><br>B}{\min }<br><br><br><br>a-b<br><br>,A=\left\{\mathrm{a1},\mathrm{\dots <br><br>}<br><br>\text{\hspace{1em}<br><br>},\mathrm{am}\right\},\mathrm{and}<br><br>\text{\hspace{1em}<br><br>}<br><br>B=\left\{\mathrm{b1},\mathrm{\dots <br><br>}<br><br>\text{\hspace{1em}<br><br>},\mathrm{bn}\right\}.$

17. The multiple person detection method according to claim 14, wherein the model image is constructed with at least one of a front model image, a left model image, and a right model image.

18. A computer-readable recording medium storing a program to execute a multiple person detection method comprising:

detecting at least one skin color region from a picked-up frame image by using skin color information;

determining whether each of the skin color regions belongs to a person candidate region; and

determining whether the skin color region belonging to the person candidate region corresponds to a person by using person shape information.