Automatic backlight compensation and exposure control

• US 7,545,435 B2
• Filed: 04/15/2005
• Issued: 06/09/2009
• Est. Priority Date: 10/15/2004
• Status: Active Grant

• Current Assignee: Lifesize, Inc. (Enghouse Systems Limited)
• Original Assignee: Lifesize, Inc. (Enghouse Systems Limited)
• Inventors: Jenkins, Michael V., Ree, Bradley Richard
• Primary Examiner(s): Ho; Tuan V
• Assistant Examiner(s): FOSSELMAN, JOEL W
• Application Number:
  US11/107,401
• Publication Number:
  US 20060082676A1
• Time in Patent Office: 1,516 Days
• Field of Search: 348/221.1, 348/223.1, 348/362, 348/363, 348/370, 358/466, 248/234, 248/235
• US Class Current: 348/362
• CPC Class Codes: H04L 12/1818 Conference organisation arrangements, e.g. handling schedules, setting up parameters needed by nodes to attend a conference, booking network resources, notifying involved parties
• CPC Class Codes: H04L 43/0811 by checking connectivity
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 9/73 Colour balance circuits, e.g. white balance circuits or colour temperature control camera processing pipelines for colour balance H04N23/88

Abstract

In some embodiments, images for the video conference may be white balanced and/or exposure controlled. White balancing may include dividing an image'"'"'s pixels into multiple ranges according to their luminance and chrominance values. In some embodiments, blue and red analog gains may be modified to bring red and blue accumulations closer to green accumulations. In some embodiments, luminance values for an image may be adjusted by increasing or decreasing exposure. The decision whether to increase or decrease the exposure may be made by looking at statistics for luminance values of skin colored regions if there are sufficient skin colored regions on the image. Otherwise, luminance values for the entire image may be analyzed to determine whether to increase or decrease exposure.

Claims

1. A method for exposure control for an image, comprising:

electronics calculating a skew of the luminance values of at least a portion of the image;

if the skew is above a predetermined value, calculating a predicted peak value according to a first formula;

if the skew is not above a predetermined value, calculating the predicted peak value according to a second formula;

if the predicted peak value is less than a predetermined low predicted peak value, increasing an exposure of the image; and

if the predicted peak value is more than a predetermined high predicted peak value, decreasing the exposure of the image.

2. The method of claim 1, wherein according to the first formula, the predicted peak value is approximately an average of the luminance values minus a standard deviation of the luminance values times a skew of the luminance values.

3. The method of claim 1, wherein according to the second formula, the predicted peak value is approximately a total of the luminance values divided by the number of luminance values.

4. The method of claim 1, wherein the skew is calculated for a portion of the image that comprises pixels of interest.

5. The method of claim 4, wherein the pixels of interest are selected from a group consisting of identified movement in the image, identified skin portions in the image, identified portions of the image closest to the camera, pixels corresponding to a speaking participant, and identified warm portions of the image.

6. A method for exposure control for an image, comprising:

electronics receiving video data for display on a display device in a videoconference, wherein the video data comprises a plurality of pixels corresponding to at least one image, wherein the at least one image comprises at least one human participant image;

determining a first plurality of pixels corresponding to a first portion of the image, wherein said determining the first plurality of pixels is performed based on identified skin portions of the at least one human participant image, wherein each of the plurality of pixels comprises a luminance value;

calculating a skew of the luminance values of the first plurality of pixels;

if the skew is above a predetermined value, calculating a predicted peak value of the luminance values according to a first formula;

if the skew is not above a predetermined value, calculating the predicted peak value of the luminance values according to a second formula;

if the predicted peak value is less than a predetermined low predicted peak value, increasing an exposure of the image; and

if the predicted peak value is more than a predetermined high predicted peak value, decreasing the exposure of the image.

7. A method for exposure control for an image, comprising:

electronics receiving video data for display on a display device in a videoconference, wherein the video data comprises a plurality of pixels corresponding to at least one image, wherein the at least one image comprises at least one human participant image;

determining a first plurality of pixels corresponding to a first portion of the image, wherein said determining the first plurality of pixels is performed based on identified skin portions of the at least one human participant image, wherein each of the plurality of pixels comprises a luminance value;

determining a percentage of the first plurality of pixels versus the plurality of pixels comprising the image;

if the percentage is above a minimum skin percentage, calculating a skew of the luminance values of the first plurality of pixels;

if the percentage is not above a minimum skin percentage, calculating a skew of the luminance values for the plurality of pixels;

if the calculated skew is above a predetermined skew value, calculating a predicted peak value according to a first formula using the luminance values used to calculate the skew;

if the skew is not above a predetermined skew value, calculating the predicted peak value according to a second formula using the luminance values used to calculate the skew;

if the predicted peak value is less than a predetermined low predicted peak value, increasing an exposure of the image; and

if the predicted peak value is more than a predetermined high predicted peak value, decreasing the exposure of the image.

8. The method of claim 7, wherein according to the first formula, the predicted peak value is approximately an average of the luminance values minus a standard deviation of the luminance values times a skew of the luminance values.

9. The method of claim 7, wherein according to the second formula, the predicted peak value is approximately a total of the luminance values divided by the number of luminance values.

10. A system, comprising:

a camera; and

electronics coupled to the camera for performing a method comprising;

receiving video data for display on a display device in a videoconference, wherein the video data comprises a plurality of pixels corresponding to at least one image, wherein the at least one image comprises at least one human participant image;

determining a first plurality of pixels corresponding to a first portion of the image, wherein said determining the first plurality of pixels is performed based on identified skin portions of the at least one human participant image, wherein each of the plurality of pixels comprises a luminance value;

determining a percentage of the first plurality of pixels versus the plurality of pixels comprising the image;

if the percentage is above a minimum skin percentage, calculating a skew of the luminance values of the first plurality of pixels;

if the percentage is not above a minimum skin percentage, calculating a skew of the luminance values for the plurality of pixels;

if the calculated skew is above a predetermined skew value, calculating a predicted peak value according to a first formula using the luminance values used to calculate the skew;

if the skew is not above a predetermined skew value, calculating the predicted peak value according to a second formula using the luminance values used to calculate the skew;

if the predicted peak value is less than a predetermined low predicted peak value, increasing an exposure of the image; and

if the predicted peak value is more than a predetermined high predicted peak value, decreasing the exposure of the image.

11. The system of claim 10, wherein according to the first formula, the predicted peak value is approximately an average of the luminance values minus a standard deviation of the luminance values times a skew of the luminance values.

12. The system of claim 11, wherein according to the second formula, the predicted peak value is approximately a total of the luminance values divided by the number of luminance values.

13. The system of claim 11, wherein the minimum skin percentage varies based on one or more image characteristics.

14. The system of claim 13, wherein the image characteristic is previous image exposure.