Systems, methods, and apparatus for exposure control

• US 8,824,827 B2
• Filed: 01/05/2012
• Issued: 09/02/2014
• Est. Priority Date: 03/17/2006
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Quan, Shuxue, Chiu, Chinchuan Andrew, Jiang, Xiaoyun
• Primary Examiner(s): Krasnic, Bernard
• Application Number:
  US13/344,514
• Publication Number:
  US 20120105675A1
• Time in Patent Office: 971 Days
• Field of Search: 382/274, 382/162, 382/165, 382/167, 382/224, 382/254, 348/221.1
• US Class Current: 382/274
• CPC Class Codes: G06V 10/26 Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: H04N 1/628 Memory colours, e.g. skin or sky
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/84 for processing colour signals

Abstract

Systems, methods and apparatus for image processing by classifying pixels are described. In some systems, an exposure control operation is performed according to the pixel classifications. In some cases, the pixels are classified according to a predetermined segmentation of a color space, based on predicted sensor responses.

Claims

1. A method of image processing, said method comprising:

capturing a raw image using an image sensor;

processing the raw image into an intermediate image, wherein said processing increases a reliability of a plurality of pixels of the intermediate image for subsequent classification;

classifying each of the plurality of pixels of the intermediate image according to a predetermined segmentation of a color space, wherein said classifying each of the plurality of pixels of the intermediate image includes calculating values of elements of a pixel classification map;

performing a noise reduction operation on the pixel classification map; and

performing an exposure control operation based on the noise-reduced pixel classification map, wherein the predetermined segmentation of the color space is based on a plurality of predicted responses of the image sensor.

2. The method of image processing according to claim 1, wherein the predetermined segmentation of the color space is based on a plurality of predicted responses of the image sensor to human skin surfaces.

3. The method of image processing according to claim 1, wherein said classifying each of the plurality of pixels according to a predetermined segmentation includes assigning each of the plurality of pixels, according to a relation between the predetermined segmentation and a color value of the pixel, to one among a first pixel class and a second pixel class, and

wherein said performing an exposure control operation includes assigning each of a plurality of blocks of the intermediate image to one among a first block class and a second block class according to a relation between (A) a number of pixels in the block that are assigned to the first pixel class and (B) a threshold value, and

wherein said performing an exposure control operation includes calculating an exposure control factor based on an average luminance value of the blocks assigned to the first block class.

4. The method of image processing according to claim 1, wherein said performing an exposure control operation includes calculating an exposure control factor based on a result of said classifying each of the plurality of pixels and changing, according to the exposure control factor, a luminance value of each of the plurality of pixels of the intermediate image.

5. The method of image processing according to claim 1, wherein said performing an exposure control operation includes:

according to a result of said classifying each of the plurality of pixels, assigning each of a plurality of blocks of the intermediate image to one among a plurality of block classes; and

based on an average luminance value of the blocks of one of the plurality of block classes, calculating an exposure control factor.

6. The method of image processing according to claim 5, wherein said performing an exposure control operation includes changing, according to the exposure control factor, a luminance value of each of the plurality of pixels of the intermediate image.

7. A computer-readable non-transitory data storage medium storing machine-executable instructions that when executed perform the method of image processing according to claim 1.

8. An apparatus comprising:

a sensor configured to capture a raw image;

a processor configured to process the raw image into an intermediate image, wherein said processing increases a reliability of a plurality of pixels of the intermediate image for subsequent classification;

a pixel classifier configured to classify, according to a predetermined segmentation of a color space, each of the plurality of pixels of the intermediate image, wherein said classifying each of the plurality of pixels of the intermediate image includes calculating values of elements of a pixel classification map;

said apparatus configured to perform a noise reduction operation on the pixel classification map; and

an exposure controller configured to perform an exposure control operation based on the noise-reduced pixel classification map, wherein the predetermined segmentation of the color space is based on a plurality of predicted responses of the sensor.

9. The apparatus according to claim 8, wherein the predetermined segmentation of the color space is based on a plurality of predicted responses of the sensor to human skin surfaces.

10. The apparatus according to claim 8, wherein said pixel classifier is configured to assign each of the plurality of pixels, according to a relation between the predetermined segmentation and a color value of the pixel, to one among a first pixel class and a second pixel class, and

wherein said exposure controller is configured to assign each of a plurality of blocks of the intermediate image to one among a first block class and a second block class according to a relation between (A) a number of pixels in the block that are assigned to the first pixel class and (B) a threshold value, and

wherein said exposure controller is configured to calculate an exposure control factor based on an average luminance value of the blocks assigned to the first block class.

11. The apparatus according to claim 8, wherein said exposure controller is configured to calculate an exposure control factor based on the pixel classifications and to change, according to the exposure control factor, a luminance value of each of the plurality of pixels of the intermediate image.

12. The apparatus according to claim 8, wherein said exposure controller includes an image block classifier configured to assign, according to the pixel classifications, each of a plurality of blocks of the intermediate image to one among a plurality of block classes, and

wherein said exposure controller includes a calculator configured to calculate, based on an average luminance value of the blocks of one of the plurality of block classes, an exposure control factor.

13. The apparatus according to claim 12, wherein said exposure controller is configured to change, according to the exposure control factor, a luminance value of each of the plurality of pixels of the intermediate image.

14. The apparatus of claim 8, wherein the apparatus comprises a camera.

15. The apparatus of claim 8, wherein the apparatus comprises a wireless communications device.

16. An image processing system, comprising:

means for processing a raw image into an intermediate image, wherein said processing increases a reliability of a plurality of pixels of the intermediate image for subsequent classification;

means for classifying each of the plurality of pixels of the intermediate image according to a predetermined segmentation of a color space, wherein said classifying each of the plurality of pixels of the intermediate image includes calculating values of elements of a pixel classification map;

means for performing a noise reduction operation on the pixel classification map; and

means for performing an exposure control operation based on the noise-reduced pixel classification map, wherein the predetermined segmentation of the color space is based on a plurality of predicted responses of an image sensor used to capture the raw image.

17. The image processing system of claim 16, wherein the predetermined segmentation of the color space is based on a plurality of predicted responses of the image sensor to human skin surfaces.

18. A method of image processing, said method comprising:

capturing a raw image using an image sensor;

processing the raw image into an intermediate image;

classifying each of a plurality of pixels of the intermediate image according to a predetermined segmentation of a color space, wherein said classifying includes calculating values of elements of a pixel classification map;

performing a noise reduction operation on the pixel classification map; and

performing an exposure control operation based on the noise-reduced pixel classification map,

wherein the predetermined segmentation of the color space is based on a plurality of predicted responses of the image sensor.

19. An apparatus comprising:

a sensor configured to capture a raw image;

a processor configured to process the raw image into an intermediate image;

a pixel classifier configured to classify, according to a predetermined segmentation of a color space, each of a plurality of pixels of the intermediate image, the pixel classifier further configured to calculate values of elements of a pixel classification map, the pixel classifier further configured to perform a noise reduction operation on the pixel classification map; and

an exposure controller configured to perform, based on the noise-reduced pixel classification map, an exposure control operation,

wherein the predetermined segmentation of the color space is based on a plurality of predicted responses of the sensor.