VIEWPOINT DETECTOR BASED ON SKIN COLOR AREA AND FACE AREA

• US 20110262001A1
• Filed: 04/22/2010
• Published: 10/27/2011
• Est. Priority Date: 04/22/2010
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Zhou, Qing, Bi, Ning, Qi, Yingyong
• Granted Patent: US 8,315,443 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

In a particular illustrative embodiment, a method of determining a viewpoint of a person based on skin color area and face area is disclosed. The method includes receiving image data corresponding to an image captured by a camera, the image including at least one object to be displayed at a device coupled to the camera. The method further includes determining a viewpoint of the person relative to a display of the device coupled to the camera. The viewpoint of the person may be determined by determining a face area of the person based on a determined skin color area of the person and tracking a face location of the person based on the face area. One or more objects displayed at the display may be moved in response to the determined viewpoint of the person.

Claims

1. A method comprising:

receiving image data corresponding to an image captured by a camera, the image including at least one object to be displayed at a device coupled to the camera;

determining a viewpoint of a person relative to a display of the device coupled to the camera, wherein determining the viewpoint comprises determining a face area of the person based on a determined skin color area of the person and tracking a face location of the person based on the face area; and

moving the at least one object displayed at the display in response to the determined viewpoint of the person.

2. The method of claim 1, wherein the skin color area of the person is determined based on a chrominance mean value and a chrominance variance value corresponding to a skin color.

3. The method of claim 2, further comprising performing a user recognition training process to determine the chrominance mean value and the chrominance variance value skin color of the person.

4. The method of claim 1, wherein detecting the viewpoint further comprises detecting eyes within the face area to confirm a location of a face within the face area.

5. The method of claim 1, wherein determining the viewpoint further comprises selecting a first area of the image to be processed based on a prior location of a face.

6. The method of claim 5, wherein in response to no face being detected within the selected area, a second area larger than the first area is selected to be processed.

7. The method of claim 1, wherein determining the viewpoint further comprises identifying pixels having a chrominance value within a skin color range.

8. The method of claim 1, wherein tracking the face location includes estimating a center pixel location of the face area of the person by:

in response to a user recognition training mode;

scanning a candidate area of the image data related to the face area of the person;

for each pixel that is within a skin tone range, accumulating sample pixel data;

after scanning all pixels in the candidate area;

calculating skin tone thresholds; and

calculating a mean value and a variance of a measured skin tone of the sampled pixel data;

for each pixel sample within the skin tone thresholds, selectively accumulating likelihood values by;

measuring a distance of the sample pixel to the mean value;

comparing the distance to the variance;

when the distance is within the variance;

accumulating horizontal and vertical likelihood values;

upon completing the selective accumulation of the likelihood values;

configuring dynamic thresholds to find peak segments in each of the horizontal and vertical likelihood values;

detecting a peak horizontal segment and a peak vertical segment based on evaluation of the dynamic thresholds; and

determining an estimated center pixel location based on the peak horizontal segment and the peak vertical segment.

9. The method of claim 8, further comprising resetting the mean value and the variance of the measured skin tone of the sampled pixel data.

10. The method of claim 8, further comprising verifying that the center pixel location is within the face area of the person.

11. The method of claim 8, further comprising initially searching the candidate area based on a prior location of the face.

12. The method of claim 8, wherein the skin tone thresholds are associated with a particular person.

13. An apparatus comprising:

a skin color detector configured to detect a skin color area of a person within image data corresponding to an image captured by a camera;

a face detector configured to detect a face area of the image based on the detected skin color area of the person; and

a viewpoint detector configured to detect a viewpoint of the person relative to a display of a device coupled to the camera based at least in part on the detected skin color area, wherein the viewpoint detector is further configured to track a face location of the person based on the detected face area;

wherein an object within the image is moved in response to the detected viewpoint of the person.

14. The apparatus of claim 13, wherein the skin color area is detected based on a chrominance mean value and a chrominance variance value corresponding to a skin color.

15. The apparatus of claim 13, further comprising the display and the camera, and wherein the skin color detector is within an image processing pipeline coupled to receive the image data from the camera and to provide display content to the display.

16. The apparatus of claim 13, wherein tracking the face location includes estimating a center pixel location of the face area of the person by:

in response to a user recognition training mode;

scanning a candidate area of image data related to the face area of the person;

for each pixel that is within a skin tone range, accumulating sample pixel data;

after scanning all pixels in the candidate area;

calculating skin tone thresholds; and

calculating a mean value and a variance of a measured skin tone of the sampled pixel data;

for each pixel sample within the skin tone thresholds, selectively accumulating likelihood values by;

measuring a distance of the sample pixel to the mean value;

comparing the distance to the variance;

when the distance is within the variance;

accumulating horizontal and vertical likelihood values;

upon completing the selective accumulation of the likelihood values;

configuring dynamic thresholds to find peak segments in each of the horizontal and vertical likelihood values;

detecting a peak horizontal segment and a peak vertical segment based on evaluation of the dynamic thresholds; and

determining an estimated center pixel location based on the peak horizontal segment and the peak vertical segment.

17. A computer readable tangible medium storing code executable by a computer, the code comprising:

code that is executable by the computer to receive image data corresponding to an image captured by a camera;

code that is executable by the computer to determine a viewpoint of a person relative to a display of a device coupled to the camera, wherein the code to determine a viewpoint of the person comprises code that is executable by the computer to determine a face area of the person based on a determined skin color area of the person and code that is executable by the computer to track a face location of the person based on the face area; and

code that is executable by the computer to move an object within the image in response to the detected viewpoint of the person.

18. The computer readable tangible medium of claim 17, further comprising code that is executable by the computer to perform a user recognition training process to determine a chrominance mean value and a chrominance variance value skin color of the person based on a skin color of the person.

19. The computer readable tangible medium of claim 17, wherein the code to determine the viewpoint of the person further comprises code that is executable by the computer to detect eyes within the face area to confirm a location of a face within the face area.

20. The computer readable tangible medium of claim 17, further comprising code for estimating a center pixel location of the face area of the person by:

in response to a user recognition training mode;

scanning a candidate area of image data related to the face area of the person;

accumulating sample pixel data for each pixel that is within a skin tone range;

after scanning all pixels in the candidate area;

calculating skin tone thresholds; and

calculating a mean value and a variance of a measured skin tone of the sampled pixel data;

for each pixel sample within the skin tone thresholds, selectively accumulating likelihood values by;

measuring a distance of the sample pixel to the mean value;

comparing the distance to the variance;

when the distance is within the variance;

accumulating horizontal and vertical likelihood values;

upon completing the selective accumulation of the likelihood values;

configuring dynamic thresholds to find peak segments in each of the horizontal and vertical likelihood values;

detecting a peak horizontal segment and a peak vertical segment based on evaluation of the dynamic thresholds; and

determining an estimated center pixel location based on the peak horizontal segment and the peak vertical segment.

21. The computer readable tangible medium of claim 20, further comprising code that is executable by the computer to reset the mean value and the variance of the measured skin tone of the sampled pixel data.

22. The computer readable tangible medium of claim 20, further comprising code that is executable by the computer to verify that the center pixel location is within the face area of the person.

23. The computer readable tangible medium of claim 20, further comprising code that is executable by the computer to initially search the candidate area based on a prior location of the face.

24. An apparatus comprising:

means for receiving image data corresponding to an image captured by a camera;

means for determining a viewpoint of a person relative to a display of a device coupled to the camera, wherein the means for determining the viewpoint comprises means for determining a face area of the person based on a determined skin color area of the person and means for tracking a face location of the person based on the face area; and

means for moving an object within the image in response to the detected viewpoint of the person.

25. The apparatus of claim 24, wherein the skin color area of the person is determined based on a chrominance mean value and a chrominance variance value corresponding to a skin color.

26. The apparatus of claim 24, wherein the means for tracking the face location includes estimating a center pixel location of the face area of the person by:

in response to a user recognition training mode;

scanning a candidate area of image data related to the face area of the person;

for each pixel that is within a skin tone range, accumulating sample pixel data;

after scanning all pixels in the candidate area;

calculating skin tone thresholds; and

calculating a mean value and a variance of a measured skin tone of the sampled pixel data;

for each pixel sample within the skin tone thresholds, selectively accumulating likelihood values by;

measuring a distance of the sample pixel to the mean value;

comparing the distance to the variance;

when the distance is within the variance;

accumulating horizontal and vertical likelihood values;

upon completing the selective accumulation of the likelihood values;

configuring dynamic thresholds to find peak segments in each of the horizontal and vertical likelihood values;

detecting a peak horizontal segment and a peak vertical segment based on evaluation of the dynamic thresholds; and

determining an estimated center pixel location based on the peak horizontal segment and the peak vertical segment.