Automatic gain and exposure control using region of interest detection

• US 8,165,416 B2
• Filed: 06/29/2007
• Issued: 04/24/2012
• Est. Priority Date: 06/29/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Cutler, Ross G.
• Primary Examiner(s): ENTEZARI, MICHELLE M
• Application Number:
  US11/771,802
• Publication Number:
  US 20090003678A1
• Time in Patent Office: 1,761 Days
• Field of Search: 382/132, 382/274
• US Class Current: 382/274
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/76 by influencing the image signals
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display

Abstract

A region of interest may be determined using any or all of sound source location, multi-person detection, and active speaker detection. An weighted mean may be determined using the region of interest and a set of backlight weight regions, or, only the set of backlight weight regions if a region of interest could not be found. The image mean is compared to a target value to determine if the image mean is greater than or less than the target value within a predetermined threshold. If the image mean is greater than the predetermined target value and predetermined threshold value, the gain and exposure are decreased. If the image mean is lesser than the predetermined target value minus the predetermined threshold value, the gain and exposure are decreased.

Claims

1. A method, comprising:

determining a region of interest in a digital image having backlight weighted regions using any selected from the list of sound source localization, multi-person detection, active speaker detection, wherein each of the backlight weighted regions are weighted differently depending on a likelihood of the backlight weighted region including a human speaker; and

determining an image mean using at least one backlight weighted region in response to determining the region of interest is empty;

decreasing gain and/or decreasing exposure in response to determining the image mean is greater than a first predetermined target; and

increasing gain and/or increasing exposure in response to determining the image mean is less than a second predetermined target subtracted from a second predetermined threshold.

2. The method of claim 1, wherein the determining a weighted image mean is performed using one or more vertically stacked backlight weighted regions, wherein each of the one or more backlight weighted regions is a horizontally oriented rectangle, wherein the pixels associated with backlight weighted regions located at the center are weighted with highest values, the pixels associated with backlight weighted regions located at the top are weighted with second highest values, and pixels associated with backlight weighted regions located at the bottom regions are weighted with third highest values.

3. The method of claim 1, wherein the first predetermined target includes an added first predetermined threshold.

4. The method of claim 1, further comprising using a tunable parameter in adjusting gain and/or exposure.

5. The method of claim 1, wherein the backlight weighted regions are weighted differently depending on a likelihood of the backlight weighted region including a human speaker.

6. The method of claim 1, wherein the region of interest is one or more pixels in the digital image including the image of a human face.

7. The method of claim 1, wherein the image mean is determined by calculating A*mean(image*backlight)+(1−

A)*mean(ROI).

8. The method of claim 1, wherein the image mean is determined using a tunable parameter with a value of 0.7.

13. The method of claim 1, wherein the image mean is determined by calculating mean(image*backlight).

9. A computer-readable memory storing instructions for performing steps when executed by a processor, comprising:

determining a region of interest in a digital image having backlight weighted regions using any selected from the list of sound source localization, multi-person detection, active speaker detection, wherein each of the backlight weighted regions are weighted differently depending on a likelihood of the backlight weighted region including a human speaker; and

determining an image mean using at least one backlight weighted region in response to determining the region of interest is empty;

decreasing gain and decreasing exposure in response to determining the image mean is greater than a first predetermined target; and

increasing gain and increasing exposure in response to determining the image mean is less than a second predetermined target subtracted from a second predetermined threshold.

10. The computer-readable memory of claim 9, wherein the determining a weighted image mean is performed using one or more vertically stacked backlight weighted regions, wherein each of the one or more backlight weighted regions is a horizontally oriented rectangle, wherein the pixels associated with backlight weighted regions located at the center are weighted with highest values, the pixels associated with backlight weighted regions located at the top are weighted with second highest values, and pixels associated with backlight weighted regions located at the bottom regions are weighted with third highest values.

11. The computer-readable memory of claim 9, wherein the first predetermined target includes an added first predetermined threshold.

12. The computer-readable memory of claim 9, wherein the second predetermined target includes a subtracted second predetermined threshold.

14. The computer-readable memory of claim 9, wherein the at least one backlight weighted region is a second backlight weighted region, and further comprising a first backlight weighted region and a third backlight weighted region, the first backlight region located above the second backlight region and the second backlight weighted region located above the third backlight weighted region, wherein the second backlight weighted region includes a human face and the pixels associated with the second backlight weighted region include a weight value higher than the pixels associated with the first backlight weighted region and the pixels associated with the second backlight weighted region include a weight value higher than the pixels associated with the third backlight weighted region, and wherein the pixels associated with the first backlight weighted region include a weight value higher than the pixels associated with the third backlight weighted region.

15. An image gain and exposure control component, comprising:

a region of interest detector for determining a region of interest in a digital image having backlight weighted regions using any selected from the list of sound source localization, multi-person detection, active speaker detection, wherein each of the backlight weighted regions are weighted differently depending on a likelihood of the backlight weighted region including a human speaker;

and

a gain and exposure adjustor, for determining an image mean using at least one backlight weighted region in response to determining the region of interest is empty;

decreasing gain and/or decreasing exposure in response to determining the image mean is greater than a first predetermined target; and

increasing gain and/or increasing exposure in response to determining the image mean is less than a second predetermined target subtracted from a second predetermined threshold parameter.

16. The image gain and exposure control component of claim 15, wherein the region of interest detector includes a sound source localization component for locating the human speaker in the digital image using an audio probability diffusion function.

17. The image gain and exposure control component of claim 15, wherein the region of interest detector includes a multi-person detector for locating one or more human speakers in the digital image.

18. The image gain and exposure control component of claim 15, wherein the region of interest detector includes an active speaker detector that uses audio, motion, and image features to detect a human speaker in the digital image.

19. The image gain and exposure control component of claim 15, further comprising an interface for a user to control the tunable parameter.

20. The image gain and exposure control component of claim 15, wherein the backlight weighted region includes at least one predetermined region of weighted pixels where one or more human speakers are located.