User face capture in projection-based systems

• US 9,531,995 B1
• Filed: 06/22/2015
• Issued: 12/27/2016
• Est. Priority Date: 03/07/2013
• Status: Active Grant

• Current Assignee: Rawles, LLC (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Arthur, Kevin Wayne, Haskin, Menashe
• Primary Examiner(s): TRAN, QUOC DUC
• Application Number:
  US14/746,497
• Time in Patent Office: 554 Days
• Field of Search: 348/14.08, 348/14.15, 348/14.16
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

A projector based user interface unit is described for implanting an interactive workspace. The interface unit projects a graphical workspace onto a display surface and monitors user interactions with the workspace. A mirror is placed on or near the workspace to reflect the face of the user to the interface unit. The interface unit captures the mirrored face of the user and provides images of the face of the user to other users for use in conjunction with collaborative techniques such as video conferencing or content sharing.

Claims

1. A system comprising:

a processor;

a camera; and

memory, accessible by the processor and storing instructions that are executable by the processor to perform acts comprising;

causing presentation of first content on a display surface;

receiving, from the camera, an image that includes at least a portion of the display surface and a reflector having known dimensions to reflect a representation of a face of a user, the user being in a field of view of the camera;

analyzing the image;

determining a portion of the reflector associated with the face of the user; and

displaying second content associated with the face of the user.

2. The system of claim 1, the acts further comprising:

analyzing the image to detect hand-based gestures performed by the user; and

responding to the hand-based gestures.

3. The system of claim 1, further comprising a depth sensor that provides depth information associated with the image, wherein analyzing the image is further based at least in part on the depth information provided by the depth sensor.

4. The system of claim 1, wherein the known dimensions of the reflector comprise a particular shape or a particular size of the reflector.

5. The system of claim 1, wherein the reflector comprises a non-planar reflective surface and the acts further comprise processing the image to correct distortions caused by the non-planar reflective surface of the reflector.

6. The system of claim 1, wherein the reflector further comprises a mounting arrangement to automatically position the reflector based at least in part on a location of the face of the user.

7. A method implemented at least in part by a system that includes a camera, the method comprising:

causing presentation of first content on a display surface;

receiving, from the camera, an image that includes at least a portion of the display surface and a reflector having known dimensions to reflect a representation of a face of a user, the user being in a field of view of the camera;

analyzing the image;

determining a portion of the reflector associated with the face of the user; and

displaying second content associated with the face of the user.

8. The method of claim 7, further comprising:

analyzing the image to detect hand-based gestures performed by the user; and

responding to the hand-based gestures.

9. The method of claim 7, wherein the second content associated with the face of the user is a moving image.

10. The method of claim 7, further comprising:

receiving depth information associated with the image; and

detecting a position of the reflector within the image based at least in part on the depth information.

11. The method of claim 7, further comprising detecting a position of the face of the user in the image using facial recognition.

12. The method of claim 7, further comprising processing the image to correct one or more aberrations, the processing comprising correcting at least one of an image reversal, a keystone effect, or a distortion caused by a dimension of the reflector.

13. The method of claim 7, wherein the reflector comprises a non-planar reflective surface, the method further comprising processing the image to correct a distortion caused by the non-planar reflective surface of the reflector.

14. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors of a system that includes a camera, cause the one or more processors to perform acts comprising:

causing presentation of first content on a display surface;

receiving, from the camera, an image that includes at least a portion of the display surface and a reflector having known dimensions to reflect a representation of a face of a user, the user being in a field of view of the camera;

analyzing the image;

determining a portion of the reflector associated with the face of the user; and

displaying second content associated with the face of the user.

15. The one or more non-transitory computer-readable media of claim 14, the acts further comprising:

detecting at least one of a hand-based gesture or a voice-based gesture performed by the user; and

responding to the hand-based gesture or the voice-based gesture.

16. The one or more non-transitory computer-readable media of claim 14, the acts further comprising sharing at least one of the first content or the second content with another display surface associated with a different user.

17. The one or more non-transitory computer-readable media of claim 14, the acts further comprising detecting a position of the face of the user in the image.

18. The one or more non-transitory computer-readable media of claim 17, wherein the reflector is positioned adjacent to at least one of the first content or the second content.

19. The one or more non-transitory computer-readable media of claim 17, the acts further comprising processing the image to correct one or more aberrations in the image caused by the reflector.

20. The one or more non-transitory computer-readable media of claim 14, the acts further comprising detecting a position of the face of the user in the image using facial recognition.