Systems and methods for iterative adjustment of video-capture settings based on identified persona

• US 9,942,481 B2
• Filed: 10/31/2016
• Issued: 04/10/2018
• Est. Priority Date: 12/31/2013
• Status: Active Grant

• Current Assignee: Personify Incorporated (Wilson Human Capital Group, Inc.)
• Original Assignee: Personify Incorporated (Wilson Human Capital Group, Inc.)
• Inventors: Venshtain, Simion, Patel, Sanjay, Lin, Dennis J.
• Primary Examiner(s): Conner, Sean
• Assistant Examiner(s): PATEL, PINALBEN V
• Application Number:
  US15/339,274
• Publication Number:
  US 20170048440A1
• Time in Patent Office: 526 Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/76 by influencing the image signals
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

Disclosed herein are systems and methods for iterative adjustment of video-capture settings based on identified persona. In an embodiment, a method includes receiving video frames that are being captured by a video camera of an ongoing scene. The method also includes identifying a persona in one or more of the received frames at least in part by identifying, in each such frame, a set of pixels classified as being representative of the persona. The method also includes selecting, based collectively on the brightness values of the pixels in the identified set, a respective adjustment command for one or more adjustable video-capture settings of the camera, as well as outputting the selected commands to the camera for use in continuing to capture video data representative of the ongoing scene.

Claims

1. A method comprising:

receiving frames of video data that are being captured by a video camera and that are representative of an ongoing scene, the video camera having one or more adjustable video-capture settings;

identifying a persona in one or more of the received frames at least in part by identifying, in each such frame, a set of pixels classified as being representative of the persona;

selecting, based collectively on brightness values of the pixels in the identified set, a respective adjustment command for one or more of the adjustable video-capture settings; and

outputting the selected adjustment commands to the video camera for use by the video camera in continuing to capture video data representative of the ongoing scene.

2. The method of claim 1, wherein identifying a set of pixels classified as being representative of the persona in a given frame comprises:

obtaining a binary alpha mask for the given frame, the binary alpha mask having a first data value associated with each pixel that is in the given frame and that is classified as being representative of the persona, the binary alpha mask having a second data value associated with each pixel that is in the given frame and that is not classified as being representative of the persona, the second data value being different than the first; and

configuring the identified set of pixels for the given frame to include only those pixels that are associated with the first data value in the corresponding binary alpha mask.

3. The method of claim 2, wherein the first data value is 1 (one), and wherein the second data value is 0 (zero).

4. The method of claim 2, wherein obtaining the binary alpha mask for the given frame comprises generating the binary alpha mask by processing image-depth data of the given frame.

5. The method of claim 2, wherein obtaining the binary alpha mask for the given frame comprises receiving the binary alpha mask from a video processor that generated the binary alpha mask by processing image-depth data of the given frame.

6. The method of claim 1, wherein selecting, based collectively on brightness values of the pixels in the identified set, a respective adjustment command for one or more of the adjustable video-capture settings comprises:

generating a statistical model reflecting a distribution of said brightness values along a brightness spectrum; and

selecting a respective adjustment command for one or more of the adjustable video-capture settings based at least in part on the statistical model.

7. The method of claim 6, wherein the statistical model comprises a histogram.

8. The method of claim 6, wherein identifying a set of pixels classified as being representative of the persona in a given frame comprises:

generating a binary alpha mask for the given frame by processing image-depth data of the given frame, the binary alpha mask having a first data value associated with each pixel that is in the given frame and that is classified as being representative of the persona, the binary alpha mask having a second data value associated with each pixel that is in the given frame and that is not classified as being representative of the persona, the second data value being different than the first; and

configuring the identified set of pixels for the given frame to include only those pixels that are associated with the first data value in the corresponding binary alpha mask.

9. The method of claim 1, wherein the one or more adjustable video-capture settings include one or more video-capture settings selected from the group consisting of exposure, aperture, shutter speed, frame rate, gain, analog gain, digital gain, zoom, optical zoom, and digital zoom.

10. The method of claim 1, further comprising selecting, based at least in part on the pixels in the identified set, a respective adjustment command for one or more video-processing settings.

11. The method of claim 10, wherein the one or more video-processing settings include one or more video-processing settings selected from the group consisting of white balance, contrast, gain, vibrance, hue, saturation, color temperature, and sharpness.

12. The method of claim 1, wherein the video camera has an associated application programming interface (API), and wherein selecting one or more of the adjustment commands comprises selecting one or more of the adjustment commands from the API.

13. The method of claim 1, wherein at least one of the selected adjustment commands specifies a particular value for the corresponding adjustable video-capture setting.

14. The method of claim 1, wherein at least one of the selected adjustment commands specifies a relative change to the corresponding adjustable video-capture setting.

15. A video-capture controller comprising:

a frame-receiving module configured to receive frames of video data that are being captured by a video camera and that are representative of an ongoing scene, the video camera having one or more adjustable video-capture settings;

a persona-identification module configured to identify a persona in one or more of the received frames at least in part by identifying, in each such frame, a set of pixels classified as being representative of the persona;

a video-capture-adjustment module configured to select, based collectively on brightness values of the pixels in the identified set, a respective adjustment command for one or more of the adjustable video-capture settings; and

a command-output module configured to output the selected adjustment commands to the video camera for use by the video camera in continuing to capture video data representative of the ongoing scene.

16. The video-capture controller of claim 15, wherein the persona-identification module is further configured to a set of pixels classified as being representative of the persona in a given frame at least in part by:

obtaining a binary alpha mask for the given frame, the binary alpha mask having a first data value associated with each pixel that is in the given frame and that is classified as being representative of the persona, the binary alpha mask having a second data value associated with each pixel that is in the given frame and that is not classified as being representative of the persona, the second data value being different than the first; and

configuring the identified set of pixels for the given frame to include only those pixels that are associated with the first data value in the corresponding binary alpha mask.

17. The video-capture controller of claim 15, wherein the video-capture-adjustment module is further configured to select a respective adjustment command for one or more of the adjustable video-capture settings at least in part by:

generating a statistical model reflecting a distribution of said brightness values along a brightness spectrum; and

selecting a respective adjustment command for one or more of the adjustable video-capture settings based at least in part on the statistical model.

18. The video-capture controller of claim 15, wherein the one or more adjustable video-capture settings include one or more video-capture settings selected from the group consisting of exposure, aperture, shutter speed, frame rate, gain, analog gain, digital gain, zoom, optical zoom, and digital zoom.

19. The video-capture controller of claim 15, wherein the video camera has an associated application programming interface (API), and wherein selecting one or more of the adjustment commands comprises selecting one or more of the adjustment commands from the API.

20. The video-capture controller of claim 15, wherein at least one of the selected adjustment commands specifies either (i) a particular value for or (ii) a relative change to the corresponding adjustable video-capture setting.