Primary and auxiliary image capture devices for image processing and related methods

• US 10,200,671 B2
• Filed: 01/28/2013
• Issued: 02/05/2019
• Est. Priority Date: 12/27/2010
• Status: Active Grant

• Current Assignee: 3DMedia Corporation
• Original Assignee: 3DMedia Corporation
• Inventors: Dahi, Bahram, Markas, Tassos, McNamer, Michael, Boyette, Jon
• Primary Examiner(s): Hallenbeck-Huber, Jeremiah C
• Application Number:
  US13/751,203
• Publication Number:
  US 20130135445A1
• Time in Patent Office: 2,199 Days
• Field of Search: 348 47
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory

Abstract

Disclosed herein are primary and auxiliary image capture devices for image processing and related methods. According to an aspect, a method may include using primary and auxiliary image capture devices to perform image processing. The method may include using the primary image capture device to capture a first image of a scene, the first image having a first quality characteristic. Further, the method may include using the auxiliary image capture device to capture a second image of the scene. The second image may have a second quality characteristic. The second quality characteristic may be of lower quality than the first quality characteristic. The method may also include adjusting at least one parameter of one of the captured images to create a plurality of adjusted images for one of approximating and matching the first quality characteristic. Further, the method may include utilizing the adjusted images for image processing.

Claims

1. An apparatus for creating an image of a scene, the apparatus comprising:

at least two image capture devices comprising of an image sensor and other components assisting in the capture process, wherein;

the first image capture device comprising a first image-quality characteristic;

the second image capture device comprising a second image-quality characteristic, the second image-quality characteristic being of lower quality than the first image-quality characteristic;

a processor, wherein said processor is configured to;

receive image data captured by one of the two sensors;

analyze the captured image data to determine a first exposure level of the object in the scene that one of the image-capture devices is focused on;

analyze an area of the captured image data to determine whether the area is over-exposed or under-exposed;

determine a second exposure level to compensate for one of the over-exposed or under-exposed area of the scene;

the apparatus configured to capture a first image with the first image sensor at the first exposure level;

the apparatus configured to capture a second image with the second image sensor at the second exposure level;

said processor further configured to;

perform image view matching by cropping and resizing the second image to match the field of view and resolution of the first image;

perform color calibration to match the color of second image to the first image;

extracting pixel disparities between the first image and the view matched, color calibrated, second image;

create a modified second of images by utilizing higher quality pixels obtained from the first image displaced according to the extracted pixel disparities to replace corresponding pixels on the view matched, color calibrated second image where the view matched, color corrected second image does not meet predefined quality criteria, and wherein the quality criteria are one of the image sensor characteristics, the lens characteristics, the uncorrectable color differential the uncorrectable distortion or the difference in noise profiles of the two image capture devices;

use the modified second image to compensate for the underexposed or overexposed areas in the first image to create a high dynamic range image.

2. The apparatus of claim 1, wherein processor identifies distortion characteristics of the image capture devices;

and

performs a transformative procedure to correct distortion for equalizing the captured image pair.

3. The apparatus of claim 1, wherein the primary and auxiliary image capture devices are components of a mobile telephone, wherein the auxiliary image capture device is configured to face in a first position towards a user and to face in a second position towards the scene, and

wherein the auxiliary image capture device includes a mechanism for directing a path of light from the scene towards the auxiliary image capture device such that the primary image capture device and the auxiliary image capture device captures the first and second images of the scene.

4. A system for creating an image of a scene, the system comprising:

at least two image capture devices comprising of an image sensor and other components assisting in the capture process, wherein;

the first image capture device comprising a first image-quality characteristic;

the second image capture device comprising a second image-quality characteristic, the second image-quality characteristic being of lower quality than the first image-quality characteristic;

a processor, wherein said processor is configured to;

receive image data captured by one of the two sensors;

analyze the captured image data to determine a first exposure level of the object in the scene that one of the image-capture devices is focused on;

analyze an area of the captured image data to determine whether the area is over-exposed or under-exposed;

determine a second exposure level to compensate for one of the over-exposed or under-exposed area of the scene;

the system configured to capture a first image with the first image sensor at the first exposure level;

the system configured to capture a second image with the second image sensor at the second exposure level;

said processor further configured to;

perform image view matching by cropping and resizing the second image to match the field of view and resolution of the first image;

perform color calibration to match the color of second image to the first image;

extracting pixel disparities between the first image and the view matched, color calibrated;

second image;

create a modified second of images by utilizing higher quality pixels obtained from the first image displaced according to the extracted pixel disparities to replace corresponding pixels on the view matched, color calibrated second image where the view matched, color corrected second image does not meet predefined quality criteria, and wherein the quality criteria are one of the image sensor characteristics, the lens characteristics, the uncorrectable color differential the uncorrectable distortion or the difference in noise profiles of the two image capture devices;

use the modified second image to compensate for the underexposed or overexposed areas in the first image to create a high dynamic range image.

5. The system of claim 4, wherein processor identifies distortion characteristics of the image capture devices;

and

performs a transformative procedure to correct distortion for equalizing the captured image pair.

6. A non-transitory computer-readable medium comprising one or more computer-readable instructions, that when executed by at least one processor of a computing device, cause the computing device to:

capture a first image at a first image-quality characteristic;

capture a second image at a second image-quality characteristic, the second image-quality characteristic being of lower quality than the first image-quality characteristic;

receive image data captured by one of the two sensors;

analyze the captured image data to determine a first exposure level of the object in the scene that one of the image-capture devices is focused on;

analyze an area of the captured image data to determine whether the area is over-exposed or under-exposed;

determine a second exposure level to compensate for one of the over-exposed or under-exposed area of the scene;

capture a first image at the first exposure level;

capture a second image at the second exposure level;

perform image view matching by cropping and resizing the second image to match the field of view and resolution of the first image;

perform color calibration to match the color of second image to the first image;

extracting pixel disparities between the first image and the view matched, color calibrated;

second image;

create a modified second of image by utilizing higher quality pixels obtained from the first image displaced according to the extracted pixel disparities to replace corresponding pixels on the view matched, color calibrated second image where the view matched, color corrected second image does not meet predefined quality criteria, and wherein the quality criteria are one of the image sensor characteristics, the lens characteristics, the uncorrectable color differential the uncorrectable distortion or the difference in noise profiles of the two image capture devices;

use the modified second image to compensate for the underexposed or overexposed areas in the first image to create a high dynamic range image.

7. The non-transitory computer-readable medium of claim 6, further comprising identifying distortion characteristics of the image capture devices;

and performing a transformative procedure to correct distortion for equalizing the captured image pair.