Image capture device with contemporaneous reference image capture mechanism

• US 8,896,725 B2
• Filed: 06/17/2008
• Issued: 11/25/2014
• Est. Priority Date: 06/21/2007
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Bigioi, Petronel, Zamfir, Adrian, Drimbarean, Alexandru, Corcoran, Peter
• Primary Examiner(s): Haskins, Twyler
• Assistant Examiner(s): Quiett, Carramah J
• Application Number:
  US12/140,532
• Publication Number:
  US 20080316327A1
• Time in Patent Office: 2,352 Days
• Field of Search: 348/218.1, 348/36, 348/47, 348/211.11, 348/220.1, 348/239, 348/42, 348/49
• US Class Current: 348/239
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 25/44 by partially reading an SSIS array
• CPC Class Codes: H04N 25/46 by combining or binning pixels
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 5/907 using static stores, e.g. storage tubes or semiconductor memories H04N5/91 takes precedence
• CPC Class Codes: H04N 9/8042 : involving data reduction

Abstract

A hand-held or otherwise portable or spatial or temporal performance-based image capture device includes one or more lenses, an aperture and a main sensor for capturing an original main image. A secondary sensor and optical system are for capturing a reference image that has temporal and spatial overlap with the original image. The device performs an image processing method including capturing the main image with the main sensor and the reference image with the secondary sensor, and utilizing information from the reference image to enhance the main image. The main and secondary sensors are contained together within a housing.

Claims

1. A system including a hand-held or otherwise portable or a spatial or temporal performance-based image capture device, comprising:

(a) a main optical system including one or more lenses, an aperture and a main sensor for capturing an original main image;

(b) a secondary optical system including a secondary sensor that has a field of view that substantially overlaps the field of view of the main optical system and is configured for capturing one or more reference images including at least a first reference image that has temporal and spatial overlap with the original main image;

(c) a processor;

(d) one or more processor-readable non-transitory media having embedded programming code for the processor to perform a digital image processing method that comprises capturing, on the hand-held or otherwise portable or spatial or temporal performance-based image capture device, said original main image and one or more reference images, including said first reference image which has a temporal and spatial overlap with the original main image, on said main and secondary sensors, respectively, and the method further comprises enhancing the original main image based on analyzing the first reference image, wherein the enhancing comprises;

(i) assessing on the device that the original main image includes one or more defects or otherwise sub-optimal characteristics based on information, image data or meta data, or combinations of the information, image data or meta data, of the one or more reference images including said first reference image relating to the one or more defects or otherwise sub-optimal characteristics of the original main image, and

(ii) correcting on the device the one or more defects or otherwise sub-optimal characteristics in the original main image based on the information, image data or meta data, or combinations of these one or more defects or other sub-optimal characteristics, of the one or more reference images including the first reference image, wherein the correcting comprises modifying image data of the original main image to create a modified image comprising an enhanced version of the original main image;

(e) a housing coupled at least with said one or more lenses, said aperture, and said main sensor of the main optical system, and with said secondary optical system including said secondary sensor that overlaps the field of view of the main optical system.

2. The system of claim 1, wherein the housing further has coupled therein said processor and said one or more processor-readable media.

3. The system of claim 1, wherein said secondary sensor is located next to said main sensor within said housing.

4. The system of claim 1, wherein said secondary sensor has a different resolution than said main sensor.

5. The system of claim 1, wherein said secondary sensor has a different spectral sensitivity than said main sensor.

6. The system of claim 1, wherein said optical system for said secondary sensor is configured for capturing a narrower field of view than that of the main sensor.

7. The system of claim 6, wherein said different field of view includes a high resolution portion of said field of view of said main sensor.

8. The system of claim 1, wherein a slight delay is designed between capturing an image on said main sensor and on said secondary sensor.

9. The system of claim 8, wherein said slight delay is longer than a strobe discharge time.

10. The system of claim 8, wherein said slight delay is within a human blink time.

11. The system of claim 1, wherein the method further comprises rendering the modified image at a digital rendering device, display or printer, or combinations of a digital rendering device, display or printer, as output from the image capture device.

12. The system of claim 1, wherein the method further comprises analyzing said one or more reference images based on predefined criteria in comparison to said main image.

13. The system of claim 12, wherein the method further comprises, based on said analyzing, creating supplemental meta data.

14. The system of claim 13, wherein the method further comprises adding said supplemental meta data to said main image at a digital data storage location.

15. The system of claim 14, wherein the method further comprises modifying said main image using said supplemental meta data.

16. The system of claim 15, wherein the method further comprises rendering the modified image at a digital rendering device, display or printer, or combinations of the digital rendering device, display or printer, as output from the image capture device.