SYSTEMS AND METHODS FOR ITERATIVE ADJUSTMENT OF VIDEO-CAPTURE SETTINGS BASED ON IDENTIFIED PERSONA

• US 20170048440A1
• Filed: 10/31/2016
• Published: 02/16/2017
• Est. Priority Date: 12/31/2013
• Status: Active Grant

• Current Assignee: Personify Incorporated (Wilson Human Capital Group, Inc.)
• Original Assignee: Personify Incorporated (Wilson Human Capital Group, Inc.)
• Inventors: Venshtain, Simion, Patel, Sanjay, Lin, Dennis J.
• Granted Patent: US 9,942,481 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/76 by influencing the image signals
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

Disclosed herein are systems and methods for iterative adjustment of video-capture settings based on identified persona. In an embodiment, a method includes receiving video frames being captured by a video camera of an ongoing scene. The method also includes identifying a persona in one or more of the received frames at least in part by identifying, in each such frame, a set of pixels that is representative of the persona in the frame and that does not include any pixels representative of a background of the frame. The method also includes selecting, based collectively on the brightness values of the pixels in the identified set of pixels of one or more frames, an adjustment command for one or more adjustable video-capture settings of the camera, as well as outputting the selected commands to the camera for use in continuing to capture video data representative of the ongoing scene.

Claims

1. A method comprising:

receiving frames of video data that are being captured by a video camera and that are representative of an ongoing scene, the video camera having one or more adjustable video-capture settings;

identifying a persona in one or more of the received frames at least in part by identifying, in each such frame, a set of pixels that is representative of the persona in the frame and that does not include any pixels representative of a background of the frame;

selecting, based collectively on the brightness values of the pixels in the identified set of pixels of one or more respective frames, a respective adjustment command for one or more of the adjustable video-capture settings; and

outputting the selected adjustment commands to the video camera for use by the video camera in continuing to capture video data representative of the ongoing scene.

2. The method of claim 1, wherein identifying, in a given frame, the set of pixels that is representative of the persona in the given frame and that does not include any pixels representative of the background of the given frame comprises:

obtaining a binary alpha mask for the given frame, the binary alpha mask having a first data value associated with each pixel that is representative of the persona, the binary alpha mask having a second data value associated with each pixel that is in the given frame but that is not representative of the persona, the second data value being different than the first; and

configuring the identified set of pixels for the given frame to include only those pixels that are associated with the first data value in the corresponding binary alpha mask.

3. The method of claim 2, wherein the first data value is 1 (one), and wherein the second data value is 0 (zero).

4. The method of claim 2, wherein obtaining the binary alpha mask for the given frame comprises generating the binary alpha mask by processing image-depth data of the given frame.

5. The method of claim 2, wherein obtaining the binary alpha mask for the given frame comprises receiving the binary alpha mask from a video processor that generated the binary alpha mask by processing image-depth data of the given frame.

6. The method of claim 1, wherein selecting, based collectively on the brightness values of the pixels in the identified set of pixels of one or more respective frames, a respective adjustment command for one or more of the adjustable video-capture settings comprises:

generating a statistical model reflecting a distribution of said brightness values along a brightness spectrum; and

selecting a respective adjustment command for one or more of the adjustable video-capture settings based at least in part on the statistical model.

7. The method of claim 6, wherein the statistical model comprises a histogram.

8. The method of claim 6, wherein identifying, in a given frame, the set of pixels that is representative of the persona in the given frame and that does not include any pixels representative of the background of the given frame comprises:

generating a binary alpha mask for the given frame by processing image-depth data of the given frame, the binary alpha mask having a first data value associated with each pixel that is representative of the persona, the binary alpha mask having a second data value associated with each pixel that is in the given frame but that is not representative of the persona, the second data value being different than the first; and

configuring the identified set of pixels for the given frame to include only those pixels that are associated with the first data value in the corresponding binary alpha mask.

9. The method of claim 1, wherein the one or more adjustable video-capture settings include one or more video-capture settings selected from the group consisting of exposure, aperture, shutter speed, frame rate, gain, analog gain, digital gain, zoom, optical zoom, and digital zoom.

10. The method of claim 1, further comprising selecting, based at least in part on the pixels in the identified set, a respective adjustment command for one or more video-processing settings.

11. The method of claim 10, wherein the one or more video-processing settings include one or more video-processing settings selected from the group consisting of white balance, contrast, gain, vibrance, hue, saturation, color temperature, and sharpness.

12. The method of claim 1, wherein the video camera has an associated application programming interface (API), and wherein selecting one or more of the adjustment commands comprises selecting one or more of the adjustment commands from the API.

13. The method of claim 1, wherein at least one of the selected adjustment commands specifies a particular value for the corresponding adjustable video-capture setting.

14. The method of claim 1, wherein at least one of the selected adjustment commands specifies a relative change to the corresponding adjustable video-capture setting.

15. A video-capture controller comprising:

a frame-receiving module configured to receive frames of video data that are being captured by a video camera and that are representative of an ongoing scene, the video camera having one or more adjustable video-capture settings;

a persona-identification module configured to identify a persona in one or more of the received frames at least in part by identifying, in each such frame, a set of pixels that is representative of the persona in the frame and that does not include any pixels representative of a background of the frame;

a video-capture-adjustment module configured to select, based collectively on the brightness values of the pixels in the identified set of pixels of one or more respective frames, a respective adjustment command for one or more of the adjustable video-capture settings; and

a command-output module configured to output the selected adjustment commands to the video camera for use by the video camera in continuing to capture video data representative of the ongoing scene.

16. A method comprising:

receiving a video-frame sequence being captured by a video camera;

receiving a sequence of image-depth data corresponding to the video-frame sequence;

generating a series of persona alpha masks by processing the video-frame sequence and the sequence of image-depth data;

generating a sequence of histograms based on the video-frame sequence and the sequence of persona alpha masks;

selecting video-capture adjustments based on the histograms; and

transmitting the video-capture adjustments to the video camera for adjusting the continued capturing of the video-frame sequence.

17. The method of claim 16, wherein the persona alpha masks are binary, having a first data value associated with each pixel in a corresponding frame that is representative of the persona, and having a second data value associated with each pixel that is in the corresponding frame but that is not representative of the persona, the second data value being different than the first.

18. The method of claim 16, wherein the histograms reflect distributions along a brightness spectrum of pixels from the video-frame sequence that match corresponding persona alpha masks.

19. The method of claim 16, wherein selecting video-capture adjustments based on the histograms comprises time filtering the histograms to select video-capture adjustments that gradually adjust one or more video-capture settings of the video camera.

20. The method of claim 16, wherein the video-capture adjustments pertain to one or both of exposure and gain.