Image processing device, image processing method, program, and electronic device for detecting a skin region of a subject

• US 9,117,114 B2
• Filed: 05/08/2013
• Issued: 08/25/2015
• Est. Priority Date: 02/04/2010
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Mitani, Satoshi, Saijo, Nobuhiro
• Primary Examiner(s): Henn, Timothy J
• Assistant Examiner(s): PHAM, QUAN L
• Application Number:
  US13/889,995
• Publication Number:
  US 20130242075A1
• Time in Patent Office: 839 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04N 23/40 Circuit details for pick-up tubes
• CPC Class Codes: H04N 9/68 for controlling the amplitude of colour signals, e.g. automatic chroma control circuits H04N9/71, H04N9/73 take precedence; camera processing pipelines for controlling the colour saturation of colour signals H04N23/86

Abstract

An image processing device for detecting a skin region representing a skin of a subject from a pickup image obtained by imaging said subject, the image processing device includes: a first irradiating section; a second irradiating section; an image pickup section; an adjusting section; and a skin detecting section.

Claims

1. An image processing device for detecting a skin region representing a skin of a subject from a pickup image obtained by imaging the subject, the image processing device comprising:

a first irradiating device configured to irradiate the subject with light of a first wavelength;

a second irradiating device configured to irradiate the subject with light of a second wavelength different from the first wavelength;

an image pickup section configured to;

capture a first pickup image obtained by imaging the subject when the subject is irradiated with light of the first wavelength, and

capture a second pickup image obtained by imaging the subject when the subject is irradiated with light of the second wavelength;

a calculating section configured to determine a difference image by determining a difference between luminance values of corresponding pixels of the first pickup image and the second pickup image;

an adjusting section configured to adjust the intensity of irradiation light of the first wavelength from the first irradiating means for the first pickup image and adjust the intensity of irradiation light of the second wavelength from the second irradiating means for the second pickup image so that pixel values of a skin region in the difference image are at least greater than a skin detection enabling value; and

a skin detecting section configured to detect the skin region in at least one of the first pickup image and the second pickup image on a basis of the adjusted first pickup image and the adjusted second pickup image.

2. The image processing device according to claim 1, further comprising a binarizing section configured to binarize the difference image such that the skin detecting section detects the skin region based on the binarized difference image.

3. The image processing device according to claim 2, wherein the skin detecting section is configured to indicate the skin region cannot be detected when pixel values of pixels forming the binarized difference image are less than the skin detection enabling value.

4. The image processing device according to claim 3, wherein the adjusting section is configured to increase a gain of image pickup section responsive to the skin detecting section being unable to detect the skin region.

5. The image processing device according to claim 3, wherein the adjusting section is configured to increase the intensity of irradiation light of the first wavelength from the first irradiating means for the first pickup image and increase the intensity of irradiation light of the second wavelength from the second irradiating means for the second pickup image responsive to the skin detecting section being unable to detect the skin region.

6. The image processing device according to claim 1, wherein the adjusting section is configured to decrease the intensity of irradiation light of the first wavelength from the first irradiating means for the first pickup image and decrease the intensity of irradiation light of the second wavelength from the second irradiating means for the second pickup image so that pixel values of the skin region in the difference image are substantially equal to the skin detection enabling value responsive to the skin detecting section determining that the pixel values of the skin re ion in the difference image are substantially greater than the skin detection enabling value.

7. The image processing device according to claim 1, wherein the skin detection enabling value corresponds to a minimum amount of irradiation light needed for the detecting section to detect the skin region.

8. The image processing device according to claim 1, wherein the image pickup section configured to capture an outside light image and the adjusting section adjusts a gain of the image pickup section on a basis of an average value of luminance values of pixels forming the outside light image before capturing the first and second pickup images.

9. The image processing device according to claim 1, wherein the first wavelength is between 800 and 900 nanometers and the second wavelength is between 900 and 1000 nanometers.

10. An image processing method of an image processing device for detecting a skin region representing a skin of a subject from a pickup image obtained by imaging the subject, the image processing device including first irradiating means, second irradiating means, image pickup means, calculating means, adjusting means, and skin detecting means, the image processing method comprising:

irradiating, via the first irradiating means, the subject with light of a first wavelength;

irradiating, via the second irradiating means, the subject with light of a second wavelength different from the first wavelength;

capturing, via the image pickup means, a first pickup image obtained by imaging the subject when the subject is irradiated with light of the first wavelength;

capturing, via the image pickup means, a second pickup image obtained by imaging the subject when the subject is irradiated with light of the second wavelength;

determining, via the calculating means, a difference image by determining a difference between luminance values of corresponding pixels of the first pickup image and the second pickup image;

adjusting, via the adjusting means, the intensity of irradiation light of the first wavelength from the first irradiating means for the first pickup image and adjusting the intensity of irradiation light of the second wavelength from the second irradiating means for the second pickup image so that pixel values of a skin region in a difference image are at least greater than a skin detection enabling value; and

detecting, via the skin detecting means, the skin region in the first pickup image on a basis of the adjusted first pickup image and the adjusted second pickup image.

11. The method according to claim 10, further comprising:

capturing, via the image pickup means, an outside light image prior to the first and second irradiating means irradiating the subject;

determining, via the adjusting means, an average value of luminance values of pixels forming the outside light image; and

adjusting, via the adjusting means, a gain of the image pickup means so that average value of luminance is equal to or lower than half a maximum luminance value of the image processing device.

12. The method according to claim 11, responsive to determining that the skin region cannot be detected:

increasing, via the adjusting means, the gain of the image pickup means responsive to determining that the previously adjusted gain is not a maximum gain;

capturing, via the image pickup means, a second instance of the first pickup image obtained by imaging the subject when the subject is irradiated with light of the first wavelength; and

capturing, via the image pickup means, a second instance of the second pickup image obtained by imaging the subject when the subject is irradiated with light of the second wavelength.

13. The method according to claim 12, responsive to determining that the skin region cannot be detected and the gain is the maximum gain:

increasing, via the adjusting means, the intensity of irradiation light of the first wavelength from the first irradiating means for the first pickup image;

increasing, via the adjusting means, the intensity of irradiation light of the second wavelength from the second irradiating means for the second pickup image;

capturing, via the image pickup means, a second instance of the first pickup image obtained by imaging the subject when the subject is irradiated with light of the first wavelength; and

capturing, via the image pickup means, a second instance of the second pickup image obtained by imaging the subject when the subject is irradiated with light of the second wavelength.

14. The method according to claim 10, further comprising:

determining, via the skin detecting means, that the pixel values of the skin region in the difference image are substantially greater than the skin detection enabling value; and

decreasing, via the adjusting means, the intensity of irradiation light of the first wavelength from the first irradiating means for a second instance of the first pickup image and decreasing the intensity of irradiation light of the second wavelength from the second irradiating means for a second instance of the second pickup image so that pixel values of the skin region in the difference image are substantially equal to the skin detection enabling value.

15. The method according to claim 10, further comprising:

smoothing, via the adjusting means, the first and the second pickup images using a low pass filter before the calculating means determines the difference image; and

detecting, via the skin detecting means, the skin region by;

setting pixel values of the difference image that are at least equal to or greater than the skin detection enabling value to a value of one,

setting pixel values of the difference image that are less than the skin detection enabling value to a value of zero, and

determining at least a certain number of pixels in a region have a value of one.

16. A machine-accessible device having instructions stored thereon that are configured when executed to cause a machine to at least:

irradiating a subject with light of a first wavelength;

recording a first pickup image obtained by imaging the subject when the subject is irradiated with light of the first wavelength;

irradiating the subject with light of a second wavelength different from the first wavelength;

recording a second pickup image obtained by imaging the subject when the subject is irradiated with light of the second wavelength;

determining a difference image by determining a difference between luminance values of corresponding pixels of the first pickup image and the second pickup image;

adjusting the intensity of irradiation light of the first wavelength for the first pickup image and adjusting the intensity of irradiation light of the second wavelength for the second pickup image so that pixel values of a skin region in the difference image are substantially equal to a skin detection enabling value; and

detecting the skin region on a basis of the adjusted first pickup image and the adjusted second pickup image.

17. The machine-accessible device of claim 16, further comprising instructions stored thereon that are configured when executed to cause a machine to at least:

determine a difference image by determining a difference between luminance values of corresponding pixels of the first pickup image and the second pickup image; and

detect the skin region in at least one of the first pickup image and the second pickup image based on pixel values of the difference image being substantially equal or greater than the skin detection enabling value.

18. The machine-accessible device of claim 16, wherein the skin detection enabling value corresponds to a minimum amount of irradiation light needed for the detecting section to detect the skin region.

19. The machine-accessible device of claim 16, further comprising:

irradiating the subject concurrently with light of the first wavelength and the second wavelength;

applying a first filter passing light of the first wavelength to record the first pickup image; and

applying a second filter passing light of the first wavelength to record the second pickup image.