Primary and auxiliary image capture devcies for image processing and related methods

• US 8,441,520 B2
• Filed: 08/13/2012
• Issued: 05/14/2013
• Est. Priority Date: 12/27/2010
• Status: Active Grant

• Current Assignee: 3DMedia Corporation
• Original Assignee: 3DMedia Corporation
• Inventors: Dahi, Bahram, McNamer, Michael, Izzat, Izzat H., Markas, Tassos
• Primary Examiner(s): Dastouri, Mehrdad
• Assistant Examiner(s): HALLENBECK-HUBER, JEREMIAH CHARLES
• Application Number:
  US13/584,744
• Publication Number:
  US 20120314036A1
• Time in Patent Office: 274 Days
• Field of Search: 348/47, 348/50
• US Class Current: 348/47
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/20 Image signal generators
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/268 based on depth image-based rendering [DIBR]
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory

Abstract

Disclosed herein are primary and auxiliary image capture devices for image processing and related methods. According to an aspect, a method may include using primary and auxiliary image capture devices to perform image processing. The method may include using the primary image capture device to capture a first image of a scene, the first image having a first quality characteristic. Further, the method may include using the auxiliary image capture device to capture a second image of the scene. The second image may have a second quality characteristic. The second quality characteristic may be of lower quality than the first quality characteristic. The method may also include adjusting at least one parameter of one of the captured images to create a plurality of adjusted images for one of approximating and matching the first quality characteristic. Further, the method may include utilizing the adjusted images for image processing.

Claims

1. A method for using primary and auxiliary image capture devices, each including an image sensor and a lens, to perform image processing, the method comprising:

using the primary image capture device to capture a first image of a scene, the first image having a first quality characteristic;

using the auxiliary image capture device to capture a second image of the scene, the second image having a second quality characteristic, and the second quality characteristic being of lower quality than the first quality characteristic;

extracting image features using one or more pixels from the images in the captured image pair;

matching extracted image features between the images in the captured image pair;

computing a transformation to adjust at least one of the matched image features between the images in the captured image pair;

applying the transformation to at least one of the images in the captured image pair creating an adjusted image;

forming a transformed image pair by selecting two images from a second group of images including the adjusted and captured images;

utilizing the transformed image pair to create another image segment utilizing lower quality pixels obtained directly from their corresponding locations from the transformed image pair; and

utilizing the transformed image pair to create another image segment utilizing higher quality pixels obtained by utilizing disparity information and higher quality pixels from the transformed image pair to replace corresponding pixels on a lower quality image based on a determination that a corresponding image segment of the lower quality image does not meet predefined quality criteria, wherein the replaced corresponding pixels are non-occluded pixels.

2. The method of claim 1, further comprising:

utilizing the transformed image pair to create another image segment utilizing one of;

lower quality pixels obtained directly from their corresponding locations from the transformed image pair; and

higher quality pixels obtained by utilizing disparity information and higher quality pixels from the transformed image pair to replace corresponding pixels on a lower quality image in response to determining that a corresponding image segment of the lower quality image does not meet predefined quality criteria, wherein the replaced corresponding pixels are non-occluded pixels.

3. The method of claim 1, further comprising determining that the corresponding image segment of the lower quality image does not meet predefined quality criteria.

4. A system for using primary and auxiliary image capture devices, each including an image sensor and a lens, to perform image processing, the system comprising:

a non-transitory memory having stored therein computer-executable instructions;

a computer processor that executes the computer-executable instructions;

an image generator configured to;

use the primary image capture device to capture a first image of a scene, the first image having a first quality characteristic;

use the auxiliary image capture device to capture a second image of the scene, the second image having a second quality characteristic, and the second quality characteristic being of lower quality than the first quality characteristic;

extract image features using one or more pixels from the images in the captured image pair;

match extracted image features between the images in the captured image pair;

compute a transformation to adjust at least one of the matched image features between the images in the captured image pair;

apply the transformation to at least one of the images in the captured image pair creating an adjusted image;

form a transformed image pair by selecting two images from a second group of images including the adjusted and captured images;

utilize the transformed image pair to create another image segment utilizing lower quality pixels obtained directly from their corresponding locations from the transformed image pair; and

utilize the transformed image pair to create another image segment utilizing higher quality pixels obtained by utilizing disparity information and higher quality pixels from the transformed image pair to replace corresponding pixels on a lower quality image based on a determination that a corresponding image segment of the lower quality image does not meet predefined quality criteria, wherein the replaced corresponding pixels are non-occluded pixels.

5. The system of claim 4, wherein the image generate is configured to:

utilize the transformed image pair to create another image segment utilizing one of;

lower quality pixels obtained directly from their corresponding locations from the transformed image pair; and

higher quality pixels obtained by utilizing disparity information and higher quality pixels from the transformed image pair to replace corresponding pixels on a lower quality image in response to determining that a corresponding image segment of the lower quality image does not meet predefined quality criteria, wherein the replaced corresponding pixels are non-occluded pixels.

6. The system of claim 4, wherein the image generator is configured to determine that the corresponding image segment of the lower quality image does not meet predefined quality criteria.

7. A non-transitory computer program product for using primary and auxiliary image capture devices, each including an image sensor and a lens, to perform image processing, said computer program product comprising:

a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising;

computer readable program code configured to use the primary image capture device to capture a first image of a scene, the first image having a first quality characteristic;

computer readable program code configured to use the auxiliary image capture device to capture a second image of the scene, the second image having a second quality characteristic, and the second quality characteristic being of lower quality than the first quality characteristic;

computer readable program code configured to extract image features using one or more pixels from the images in the captured image pair;

computer readable program code configured to match extracted image features between the images in the captured image pair;

computer readable program code configured to compute a transformation to adjust at least one of the matched image features between the images in the captured image pair;

computer readable program code configured to apply the transformation to at least one of the images in the captured image pair creating an adjusted image;

computer readable program code configured to form a transformed image pair by selecting two images from a second group of images including the adjusted and captured images;

computer readable program code configured to utilize the transformed image pair to create another image segment utilizing lower quality pixels obtained directly from their corresponding locations from the transformed image pair; and

computer readable program code configured to utilize the transformed image pair to create another image segment utilizing higher quality pixels obtained by utilizing disparity information and higher quality pixels from the transformed image pair to replace corresponding pixels on a lower quality image based on a determination that a corresponding image segment of the lower quality image does not meet predefined quality criteria, wherein the replaced corresponding pixels are non-occluded pixels.

8. The computer program product of claim 7, further comprising computer readable program code configured to:

utilize the transformed image pair to create another image segment utilizing one of;

lower quality pixels obtained directly from their corresponding locations from the transformed image pair; and

higher quality pixels obtained by utilizing disparity information and higher quality pixels from the transformed image pair to replace corresponding pixels on a lower quality image in response to determining that a corresponding image segment of the lower quality image does not meet predefined quality criteria, wherein the replaced corresponding pixels are non-occluded pixels.

9. The computer program product of claim 7, further comprising computer readable program code configured to determine that the corresponding image segment of the lower quality image does not meet predefined quality criteria.