Skin detection in an augmented reality environment

• US 9,380,270 B1
• Filed: 08/31/2011
• Issued: 06/28/2016
• Est. Priority Date: 08/31/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Worley, William Spencer III, Yao, Ning, Crump, Edward Dietz, Gopalan, Sowmya
• Primary Examiner(s): Anyikire, Chikaodili E
• Application Number:
  US13/222,452
• Time in Patent Office: 1,763 Days
• Field of Search: 348/77
• US Class Current: 1/1
• CPC Class Codes: A61B 1/041 Capsule endoscopes for imaging
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

An augmented reality environment allows interaction between virtual and real objects. Wavelength modulated light with a known spectral signature available within the augmented reality environment is used to generate spectral reflectance data for objects within the environment. This reflectance data is used to determine whether objects or portions thereof are skin.

Claims

1. A system comprising:

a processor;

a projector coupled to the processor and configured to illuminate a scene and at least a portion of an object associated with a user with light having a known spectral signature;

a camera coupled to the processor and configured to acquire image data of the light as reflected from the object;

a memory accessible by the processor and configured to store one or more skin reflectance references; and

a skin recognition module accessible by the processor configured to;

determine a spectral reflectance of the object from the image data acquired by the camera;

compare the spectral reflectance of the object with a skin reflectance reference stored in the memory; and

determine an identity of the user based at least in part upon the spectral reflectance of the object and the skin reflectance reference stored in the memory.

2. The system of claim 1, wherein the projector comprises a digital micromirror device.

3. The system of claim 1, wherein illuminating the scene comprises illuminating the scene with structured light.

4. The system of claim 1, wherein illuminating the scene comprises illuminating the scene with a sequence of sub-frames with each sub-frame having a different spectral signature.

5. The system of claim 4, wherein illuminating the scene comprises illuminating the scene with visible light.

6. The system of claim 1, wherein the skin recognition module is further configured to determine the known spectral signature by reflectance from a reference object with known spectral reflectance.

7. The system of claim 1, wherein the skin recognition module is further configured to compensate the spectral reflectance for ambient illumination.

8. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

illuminating skin of a user with light having a known spectral signature;

acquiring spectral reflectance data resulting from the illumination;

comparing the spectral reflectance data with previously stored spectral reflectance;

determining an adjustment to the previously stored spectral reflectance based at least partly on the spectral reflectance data;

associating the adjustment with an identity of the user; and

determining the identity of the user based at least in part on the adjustment.

9. The one or more non-transitory computer-readable storage media of claim 8, wherein the light comprises ultraviolet light.

10. The one or more non-transitory computer-readable storage media of claim 8, wherein illuminating the skin of the user comprises modulating a light source to sequentially generate a plurality of known spectral signatures in differing spectral bands.

11. The one or more non-transitory computer-readable storage media of claim 8, the acts further comprising storing the spectral reflectance data in a skin reflectance reference datastore in association with the user.

12. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

illuminating at least a portion of a scene containing an object associated with a user with light of a known spectral signature;

acquiring spectral reflectance data from light reflected by the object in the scene;

comparing the spectral reflectance data with reflectance reference data; and

based at least in part upon the spectral reflectance data and the reflectance reference data, estimating an age of the user.

13. The one or more non-transitory computer-readable storage media of claim 12, wherein the acts further comprise determining that the reflectance reference data previously represented skin.

14. The one or more non-transitory computer-readable storage media of claim 12, wherein illuminating the at least the portion of the scene comprises directing a digital micromirror device to project the light with the known spectral signature, and wherein acquiring the spectral reflectance data comprises selectively imaging the scene in wavelengths corresponding to the known spectral signature.

15. The one or more non-transitory computer-readable storage media of claim 12, the acts further comprising acquiring spectral data from light emitted by the object where a spectral band of the light emitted by the object differs from a spectral band of the light illuminating the object.

16. The one or more non-transitory computer-readable storage media of claim 13, wherein the reflectance reference data is stored in a skin reflectance reference datastore that comprises a plurality of pre-determined spectral reflectance data associated with mammalian skin.

17. The one or more non-transitory computer-readable storage media of claim 12, wherein the reflectance reference data is stored in a skin reflectance reference datastore that comprises a plurality of pre-determined spectral reflectance data associated with skin.

18. The one or more non-transitory computer-readable storage media of claim 12, wherein illuminating the at least the portion of the scene comprises directing a digital micromirror device to generate an image with the light of the known spectral signature.

19. The one or more non-transitory computer-readable storage media of claim 12, wherein illuminating the at least the portion of the scene comprises illuminating the scene at a plurality of intervals, each of the plurality of intervals being illuminated with light of different spectral signatures.

20. The one or more non-transitory computer-readable storage media of claim 12, the acts further comprising:

acquiring an image of at least a portion of the scene with ambient illumination; and

compensating for the ambient illumination in the spectral reflectance data.