Real-time Visual Feedback for User Positioning with Respect to a Camera and a Display

• US 20160148384A1
• Filed: 08/28/2015
• Published: 05/26/2016
• Est. Priority Date: 11/21/2014
• Status: Active Grant

• Current Assignee: iProov Limited
• Original Assignee: iProov Limited
• Inventors: Newell, Andrew, Palmer, Joe, Bud, Andrew
• Granted Patent: US 9,412,169 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/141 Control of illumination
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/67 by interactive indications to the user
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/44 Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Systems, methods, and computer program products provide near real-time feedback to a user of a camera-enabled device to guide the user to capture self-imagery when the user is in a desired position with respect the camera and/or the display of the device. The desired position optimizes aspects of self-imagery that is captured for applications in which the imagery is not primarily intended for the user'"'"'s consumption. One class of such applications includes applications that rely on illuminating the user'"'"'s face with light from the device'"'"'s display screen. The feedback is abstracted to avoid biasing the user with aesthetic considerations. The abstracted imagery may include real-time cartoon-like line drawings of edges detected in imagery of the user'"'"'s head or face.

Claims

1. A method of capturing imagery of a user, the method comprising:

capturing positioning imagery of the user with a camera;

processing the positioning imagery substantially in real time to generate abstracted positioning imagery of the user wherein the abstracted positioning imagery represents;

at least a partial outline of a body part of the user; and

a plurality of anatomical features of the body part of the user inside and distinct from the at least partial outline;

displaying the abstracted positioning imagery of the user on a display facing the user, wherein the abstracted positioning imagery provides visual feedback for guiding the user to move to a desired position with respect to the camera, without the abstracted positioning imagery also providing sufficient visual information for the user to assess a visual quality of the positioning imagery; and

when the user is positioned in the desired position with respect to the camera, capturing application imagery of the user; and

providing the captured application imagery of the user to an application without displaying the captured application imagery to the user.

2. The method of claim 1, wherein the abstracted positioning imagery further provides visual feedback for guiding the user to move to a desired position with respect to the display, and wherein the application imagery of the user is captured when the user is positioned in the desired position with respect to the display.

3. The method of claim 1, wherein the display is capable of illuminating parts of the user that are closest to the display and included within a field of view of the camera when the user is in the desired position with respect to the camera.

4. The method of claim 1, wherein a normal to a centroid of the display intersects a head of the user at a point less than approximately two inches from a tip of a nose of the user when the user is in the desired position with respect to the camera.

5. The method of claim 1, wherein the user is located at a closest comfortable distance from the display when the user is in the desired position with respect to the camera.

6. The method of claim 1, wherein the camera is angled upwards towards the user when the user is in the desired position with respect to the camera.

7. The method of claim 1, wherein displaying the abstracted imagery includes displacing imagery of the user to appear centered in the display when the user is off-center in the captured positioning imagery.

8. The method of claim 1, wherein the captured imagery includes video imagery.

9. The method of claim 1, wherein the application imagery includes video imagery.

10. The method of claim 1, wherein the application imagery includes a still image.

11. The method of claim 1, wherein the display is used to illuminate the user during capture of the application imagery of the user.

12. The method of claim 1, wherein the camera and the display are connected to a local computer-based system having a network connection, and further comprising:

at the local computer-based system, receiving data via the network connection for controlling the display; and

during capture of the application imagery, using the local computer-based system to control the display using the data received at the computer-based system via the network connection.

13. The method of claim 12, wherein the received data cause the display to act as a source of controlled illumination of the user.

14. The method of claim 1, wherein the abstracted positioning imagery includes lines representing substantial changes in brightness at small spatial scales in captured positioning imagery of the user.

15. The method of claim 1, wherein the abstracted positioning imagery includes blurred imagery of the user.

16. The method of claim 1, wherein the abstracted positioning imagery includes circular features with colors based on colors of the captured positioning imagery.

17. The method of claim 1, wherein the user is alerted when the application imagery is captured.

18. The method of claim 1, wherein assessing a visual quality of the positioning imagery includes assessing an aesthetic quality of the positioning imagery.

19. The method of claim 1, further comprising:

after capturing application imagery of the user, displaying second abstracted positioning imagery of the user on the display facing the user, wherein the second abstracted positioning imagery provides visual feedback for guiding the user to move to a second desired position with respect to the camera;

when the user is positioned in the second desired position with respect to the camera, capturing second application imagery of the user; and

providing the second captured application imagery of the user to the application without displaying the captured application imagery to the user.

20. The method of claim 19, wherein a line connecting the first-mentioned desired position and the second desired position is substantially parallel to a plane of the display.

21. The method of claim 19, wherein a displacement between the first-mentioned desired position and the second desired position enables the application to use the first-mentioned captured application imagery and the second captured application imagery to extract three-dimensional information about the user.

22. A mobile device comprising:

a CPU; and

a memory storing instructions for execution by the CPU, wherein execution of the instructions on the CPU implements a method of capturing imagery of a user of the mobile device, the method comprising;

capturing positioning imagery of the user with a camera;

processing the positioning imagery substantially in real time to generate abstracted positioning imagery of the user wherein the abstracted positioning imagery represents;

at least a partial outline of a body part of the user; and

a plurality of anatomical features of the body part of the user inside and distinct from the at least partial outline;

displaying the abstracted positioning imagery of the user on a display facing the user, wherein the abstracted positioning imagery provides visual feedback for guiding the user to move to a desired position with respect to the camera, without the abstracted positioning imagery also providing sufficient visual information for the user to assess a visual quality of the positioning imagery;

when the user is positioned in the desired position with respect to the camera, capturing application imagery of the user; and

providing the captured application imagery of the user to an application without displaying the captured application imagery to the user.

23. A method of capturing imagery of a user, the method comprising:

capturing positioning imagery of the user with a camera;

processing the positioning imagery substantially in real time to generate abstracted imagery of the user;

displaying the abstracted imagery of the user on a display facing the user, wherein the abstracted imagery provides visual feedback for guiding the user to move to a desired position with respect to the display, without the abstracted imagery also providing sufficient visual information for the user to assess a visual quality of the captured positioning imagery; and

when the user is positioned in the desired position with respect to the display, capturing application imagery of the user; and

providing the captured application imagery of the user to an application without displaying the captured application imagery to the user.

24. The method of claim 23, wherein the abstracted imagery is based in part on a position of the camera with respect to the display.

25. The method of claim 24, wherein the position of the camera with respect to the display is inferred from analyzing the captured positioning imagery to determine at least one of a pose and a gaze of the user.

26. The method of claim 24, wherein the position of the camera with respect to the display is inferred from analyzing the captured positioning imagery when the user is directed to look at a displayed visual element on the display.