Local adaptive histogram equalization

• US 10,013,764 B2
• Filed: 06/19/2014
• Issued: 07/03/2018
• Est. Priority Date: 06/19/2014
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Atanassov, Kalin Mitkov, Nash, James Wilson, Verrall, Stephen Michael, Siddiqui, Hasib Ahmed
• Primary Examiner(s): Senfi, Behrooz
• Application Number:
  US14/309,458
• Publication Number:
  US 20150371387A1
• Time in Patent Office: 1,475 Days
• Field of Search: 348 49, 382162
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 23/957 Light-field or plenoptic cameras or camera modules

Abstract

Certain aspects relate to systems and techniques for performing local intensity equalization on images in a set of images exhibiting local intensity variations. For example, the local intensity equalization can be used to perform accurate region matching and alignment of the images. The images can be partitioned into regions of pixel blocks, for instance based on location, shape, and size of identified keypoints in the images. Regions depicting the same feature in the images can be equalized with respect to intensity. Region matching based on the keypoints in the intensity-equalized regions can be performed with accuracy even in images captured by asymmetric sensors or exhibiting spatially varying intensity.

Claims

1. An image capture device comprising:

a first image sensor;

a second image sensor; and

an image processor configured to;

receive image data including a first image from the first image sensor and a second image from the second image sensor, the first image and the second image depicting at least a portion of an image scene and exhibiting local intensity variation relative to one another,

partition the first image into a plurality of first blocks of pixels,

partition the second image into a plurality of second blocks of pixels,

determine a pair of corresponding blocks, the pair of corresponding blocks including an identified first block of the plurality of first blocks and an identified second block of the plurality of second blocks, and

generate equalized local intensity values for pixels included in the identified first block of the first image and the identified second block of the second image, including mapping intensity values of pixels in the first block to intensity values of pixels in the second-block such that a histogram of the intensity values of pixels of the first block are transformed to more closely resemble a histogram of the intensity values of the pixels of the second block.

2. The device of claim 1, wherein the first image sensor has a first intensity response and wherein the second image sensor has a second intensity response, wherein the first intensity response is different from the second intensity response.

3. The device of claim 1, wherein the first image and the second image exhibiting the local intensity variation is based at least partly on the first image comprising a NIR image and the second image comprising a RGB image.

4. The device of claim 1, wherein the first image and the second image exhibiting the local intensity variation is based at least partly on a first lighting condition of the image scene during capture of the first image being different from a second lighting condition of the image scene during capture of the second image.

5. The device of claim 1, further comprising a matching module including instructions that configure the image processor to perform region matching based at least partly on the equalized local intensity values for the pixels included in the identified first block and the identified second block.

6. The device of claim 5, further comprising a depth map construction module including instructions that configure the image processor to generate a depth map based at least partly on a result of the region matching.

7. The device of claim 1, further comprising a feature detection module including instructions that configure the image processor to identify a plurality of keypoints in the first image and to identify a plurality of keypoints in the second image.

8. The device of claim 7, further comprising a key region determination module including instructions that configure the image processor to partition the first image based at least partly on the plurality of keypoints in the first image and to partition the second image based at least partly on the plurality of keypoints in the second image.

9. The device of claim 1, wherein the first image sensor and the second image sensor are configured as at least a portion of one of a stereoscopic imaging system or multispectral imaging system.

10. A method for processing a set of images, the method comprising:

receiving image data representing a first image of an image scene;

receiving image data representing a second image of at least a portion of the image scene, the second image exhibiting variations in local intensity values relative to local intensity values of the first image;

partitioning each of the first image and the second image into a plurality of blocks of pixels;

determining a pair of corresponding blocks, the pair of corresponding blocks including a first block of the plurality of blocks of pixels and a second block of the plurality of blocks of pixels, the first block included in the image data representing the first image and the second block included in the image data representing the second image; and

generating equalized local intensity values for pixels included in the first block of the first image and pixels included in the second block of the second image comprising generating a mapping of intensity values of pixels in the first block to intensity values of pixels in the second block such that a histogram of the intensity values of pixels of the first block are transformed to more closely resemble a histogram of the intensity values of the pixels of the second block.

11. The method of claim 10, further comprising receiving the first image and the second image from a pair of asymmetric sensors.

12. The method of claim 10, further comprising receiving the first image from a NIR image sensor and receiving the second image from a RGB image sensor.

13. The method of claim 10, wherein generating equalized local intensity values comprises determining a probability mass function for each of the first block and the second block.

14. The method of claim 13, wherein generating equalized local intensity values comprises summing mass values in the probability mass function for each of the first block and the second block to compute a cumulative mass function for each of the first block and the second block.

15. The method of claim 14, wherein generating equalized local intensity values comprises using the computed cumulative mass function for each of the first block and the second block to generate the mapping of intensity values of pixels in the first block to intensity values of pixels in the second block.

16. The method of claim 15, wherein generating equalized local intensity values further comprises performing bilinear interpolation on the mapping of the intensity values to generate a new luma value for each of the pixels in the first block.

17. The method of claim 10, further comprising identifying a plurality of keypoints in each of the first image and the second image.

18. The method of claim 17, wherein partitioning each of the first image and the second image into a plurality of blocks of pixel is based at least partly on one or more of a location, size, shape, and number of the plurality of keypoints.

19. The method of claim 18, wherein determining the pair of corresponding blocks comprises determining that the first block and the second block depict the same keypoint of the plurality of keypoints.

20. The method of claim 10, further comprising performing region matching based at least partly on the equalized local intensity values.

21. The method of claim 20, further comprising constructing a depth map based at least partly on a result of the region matching.

22. Non-transitory computer-readable medium storing instructions that, when executed, cause an image processor to perform operations comprising:

receiving image data representing two or more images captured by a first image sensor and a second image sensor;

partitioning each of the two or more images into a plurality of blocks of pixels;

determining a pair of corresponding blocks, the pair of corresponding blocks including a first block of the plurality of blocks of pixels and a second block of the plurality of blocks of pixels, the first block included in a first image or the two or more images and the second block included in a second image of the two or more images; and

generating equalized intensity values for pixels included in the first block of the first image and pixels included in the second block of the second image, including mapping intensity values of pixels in the first block to intensity values of pixels in the second block such that a histogram of the intensity values of pixels of the first block are transformed to more closely resemble a histogram of the intensity values of the pixels of the second block.

23. The non-transitory computer-readable medium of claim 22, the operations further comprising identifying a plurality of keypoints in each of the first image and the second image.

24. The non-transitory computer-readable medium of claim 22, the operations further comprising performing region matching based at least partly on the equalized intensity values.

25. The non-transitory computer-readable medium of claim 24, the operations further comprising constructing a stereoscopic depth map based at least partly on a result of the region matching.

26. An imaging apparatus comprising:

means for receiving image data representing a first image of an image scene;

means for receiving image data representing a second image of at least a portion of the image scene, the second image exhibiting variations in local intensity values relative to local intensity values of the first image;

means for partitioning each of the first image and the second image into a plurality of blocks of pixels;

means for determining a pair of corresponding blocks, the pair of corresponding blocks including a first block of the plurality of blocks of pixels and a second block of the plurality of blocks of pixels, the first block included in the image data representing the first image and the second block included in the image data representing the second image; and

means for generating equalized local intensity values for pixels included in the first block and pixels included in the second block, the means for generating equalized local intensity values comprising means for generating a mapping of intensity values of pixels in the first block to intensity values of pixels in the second block such that a histogram of the intensity values of pixels of the first block are transformed to more closely resemble a histogram of the intensity values of the pixels of the second block.

27. The apparatus of claim 26, wherein the means for receiving image data representing the first image comprises a NIR sensor and the means for receiving image data representing the second image comprises a RGB sensor.

28. The apparatus of claim 26, wherein the means for receiving image data representing the first image and the means for receiving image data representing the second image comprises a pair of asymmetric image sensors.

29. The apparatus of claim 26, wherein the means for receiving image data representing the first image and the means for receiving image data representing the second image comprises a single image sensor.

30. The apparatus of claim 29, wherein the single image sensor captures the first image and the second image under different lighting conditions.