AUTOMATIC GAIN AND EXPOSURE CONTROL USING REGION OF INTEREST DETECTION

• US 20090003678A1
• Filed: 06/29/2007
• Published: 01/01/2009
• Est. Priority Date: 06/29/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Cutler, Ross G.
• Granted Patent: US 8,165,416 B2
• Time in Patent Office: Days
• US Class Current: 382/132
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/76 by influencing the image signals
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display

Abstract

A region of interest may be determined using any or all of sound source location, multi-person detection, and active speaker detection. An weighted mean may be determined using the region of interest and a set of backlight weight regions, or, only the set of backlight weight regions if a region of interest could not be found. The image mean is compared to a target value to determine if the image mean is greater than or less than the target value within a predetermined threshold. If the image mean is greater than the predetermined target value and predetermined threshold value, the gain and exposure are decreased. If the image mean is lesser than the predetermined target value minus the predetermined threshold value, the gain and exposure are decreased.

Claims

1. A method, comprising:

determining a region of interest in a digital image using any selected from the list of sound source localization, multi-person detection, active speaker detection; and

determining an exposure/gain metric using the region of interest in response to determining the region of interest is non-empty;

decreasing gain and/or decreasing exposure in response to determining the image mean is greater than a first predetermined target; and

increasing gain and/or increasing exposure in response to determining the image mean Is less than a second predetermined target.

2. The method of claim 1, wherein the region of interest includes an area weighted more heavily if the area is determined to include a human face that is speaking.

3. The method of claim 1, wherein the region of interest includes an area weighted less heavily if the area is determined to include a human face that is not speaking.

4. The method of claim 1, wherein the exposure/gain metric is determined using a tunable parameter.

5. The method of claim 1, wherein the weighted image mean is determined using at least one backlight weighted region.

6. The method of claim 1, wherein the region of interest is one or more pixels in the digital image including the image of a human face.

7. The method of claim 1, wherein the gain/exposure metric is determined by calculating A*mean(image*backlight)+(1−

A)*mean(ROI).

8. The method of claim 1, wherein the gain/exposure metric is determined using a tunable parameter with a value of 0.7.

13. The method of claim 1, wherein the image mean is determined by calculating mean(image*backlight).

9. One or more device-readable media having device-executable instructions for performing steps comprising:

determining a region of interest in a digital image using any selected from the list of sound source localization, multi-person detection, active speaker detection; and

determining an image mean using at least one backlight weighted region in response to determining the region of interest is empty;

decreasing gain and decreasing exposure in response to determining the image mean is greater than a first predetermined target; and

increasing gain and increasing exposure in response to determining the image mean Is less than a second predetermined target subtracted from a second predetermined threshold.

10. The one or more device-readable media of claim 9, wherein the determining a weighted image mean is performed using one or more vertically stacked backlight weighted regions, wherein each of the one or more backlight weighted regions is a horizontally oriented rectangle, wherein the pixels associated with backlight weighted regions located at the center are weighted with highest values, the pixels associated with backlight weighted regions located at the top are weighted with second highest values, and pixels associated with backlight weighted regions located at the bottom regions are weighted with third highest values.

11. The one or more device-readable media of claim 9, wherein the first predetermined target includes an added first predetermined threshold.

12. The one or more device-readable media of claim 9, wherein the second predetermined target includes a subtracted second predetermined threshold.

14. The one or more device-readable media of claim 9, wherein the at least one backlight weighted region is a second backlight weighted region, and further comprising a first backlight weighted region and a third backlight weighted region, the first backlight region located above the second backlight region and the second backlight weighted region located above the third backlight weighted region, wherein the second backlight weighted region includes a human face and the pixels associated with the second backlight weighted region include a weight value higher than the pixels associated with the first backlight weighted region and the pixels associated with the second backlight weighted region include a weight value higher than the pixels associated with the third backlight weighted region, and wherein the pixels associated with the first backlight weighted region include a weight value higher than the pixels associated with the third backlight weighted region.

15. An image gain and exposure control component, comprising:

a region of interest detector, for determining an area of pixels in a digital image where at least one human speaker is located;

a backlight weighted region, for providing at least one area of pixels having a weight;

a tunable parameter; and

a gain and exposure adjustor, for increasing or decreasing gain and increasing or decreasing exposure in response to calculating an image mean using the region of interest, the backlight value mask, and the tunable parameter.

16. The image gain and exposure control component of claim 15, wherein the region of interest detector includes a sound source localization component for locating a human speaker in the digital image using an audio probability diffusion function.

17. The image gain and exposure control component of claim 15, wherein the region of interest detector includes a multi-person detector for locating one or more human speakers in the digital image.

18. The image gain and exposure control component of claim 15, wherein the region of interest detector includes an active speaker detector that uses audio, motion, and image features to detect a human speaker in the digital image.

19. The image gain and exposure control component of claim 15, further comprising an interface for a user to control the tunable parameter.

20. The image gain and exposure control component of claim 15, wherein the backlight weighted region includes at least one predetermined region of weighted pixels where one or more human speakers are located.