METHOD AND APPARATUS FOR OPERATING A DUAL LENS CAMERA TO AUGMENT AN IMAGE

• US 20080218611A1
• Filed: 03/09/2007
• Published: 09/11/2008
• Est. Priority Date: 03/09/2007
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Parulski, Kenneth A., Border, John N., Fredlund, John R., Manico, Joseph A., Parada, Robert J., Janson, Wilbert F.
• Granted Patent: US 7,859,588 B2
• Time in Patent Office: Days
• US Class Current: 348/262
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

An electronic camera for producing an output image of a scene from a captured image signal includes a first imaging stage comprising a first image sensor for generating a first sensor output and a first lens for forming a first image of the scene on the first image sensor, and a second imaging stage comprising a second image sensor for generating a second sensor output and a second lens for forming a second image of the scene on the second image sensor. The sensor output from the first imaging stage is used as a primary output image for forming the captured image signal and the sensor output from the second imaging stage is used as a secondary output image for modifying the primary output image, thereby generating an enhanced, captured image signal.

Claims

1. An electronic camera for producing an output image of a scene from a captured image signal, said electronic camera comprising:

a first imaging stage comprising a first image sensor for generating a first sensor output and a first lens for forming a first image of the scene on the first image sensor;

a second imaging stage comprising a second image sensor for generating a second sensor output and a second lens for forming a second image of the scene on the second image sensor; and

a processing stage for using the sensor output from the first imaging stage as a primary output image for forming the captured image signal and using the sensor output from the second imaging stage as a secondary output image for modifying the primary output image, thereby generating an enhanced, captured image signal.

2. The electronic camera as claimed in claim 1 wherein the secondary output image is used by the processing stage to sharpen portions of the primary image.

3. The electronic camera as claimed in claim 2 wherein the secondary output image is used by the processing stage to extend the depth of field of the primary still image.

4. The electronic camera as claimed in claim 2 wherein the secondary output image is used by the processing stage to selectively sharpen portions of the primary still image and defocus other portions of the image.

5. The electronic camera as claimed in claim 1 wherein the secondary output image is captured by the second imaging stage at a different focus position than used by the first imaging stage to capture the primary output image.

6. The electronic camera as claimed in claim 1 wherein the secondary output image is used by the processing stage to modify the dynamic range of the primary image.

7. The electronic camera as claimed in claim 1 wherein the secondary output image is captured by the second imaging stage at a different exposure level than used by the first imaging stage to capture the primary output image.

8. The electronic camera as claimed in claim 1 wherein the primary and secondary output images are submitted to an external processor and the secondary output image is used by the external processor to modify the primary output image.

9. The electronic camera as claimed in claim 1 wherein the captured image signal is a still image signal and the secondary output image modifies at least one of the depth of field and the dynamic range of the still image signal.

10. The electronic camera as claimed in claim 1 wherein the captured image signal is a video image signal and the secondary output image modifies at least one of the depth of field and the dynamic range of the video image signal.

11. The electronic camera as claimed in claim 1 wherein the imaging stages are set for different exposure times so that different levels of noise and motion blur are present in the respective primary and secondary output images, whereby a modified primary image is then formed by replacing portions of the primary image with portions of the secondary image such that the level of motion blur is reduced while minimizing an increase in the noise level.

12. An electronic camera for producing an output image of a scene from a captured image signal, said electronic camera comprising:

a first imaging stage comprising a first image sensor for generating a first sensor output, a first lens for forming a first image of the scene on the first image sensor, and a first lens focus adjuster for adjusting focus of the first lens responsive to a first focus detection signal;

a second imaging stage comprising a second image sensor for generating a second sensor output, a second lens for forming a second image of the scene on the second image sensor, and a second lens focus adjuster for adjusting focus of the second lens responsive to a second focus detection signal; and

a processing stage for (a) selecting the sensor output from the first imaging stage as a primary output image for forming the captured image signal and using the sensor output from one or both of the first and second imaging stages to focus the first lens at a primary focus distance to provide the primary output image, and (b) selecting the sensor output from the second imaging stage as a secondary output image and using the sensor output from the second imaging stage to generate the second focus detection signal to focus the second lens at a secondary focus distance to provide a secondary still image, whereby the secondary output image is used to modify the primary output image and generate an enhanced captured image signal.

13. The electronic camera as claimed in claim 12 wherein the secondary output image is used to sharpen portions of the primary still image that are positioned near the secondary focus distance.

14. The electronic camera as claimed in claim 13 wherein the processing stage uses the secondary output image to sharpen portions of the primary still image that are positioned near the secondary focus distance.

15. The electronic camera as claimed in claim 13 wherein the primary and secondary output images are submitted to an external processor and the secondary output image is used by the external processor to sharpen portions of the primary still image that are positioned near the secondary focus distance.

16. The electronic camera as claimed in claim 12 wherein the processing stage selects either (a) the sensor output from the first imaging stage as the primary output image and uses the sensor output from the second imaging stage to generate the secondary output image and the first focus detection signal for the selected primary imaging stage or (b) the sensor output from the second imaging stage as the primary output image and uses the sensor output from the first imaging stage to generate the secondary output image and the second focus detection signal for the selected primary imaging stage, whereby the focus detection signal is applied to the lens focus adjuster of the selected primary imaging stage in order to adjust the focus of the image providing the sensor output for the primary output image.

17. A method for operating a digital camera having first and second imaging stages for forming separate images of a scene, said method comprising the steps of:

selecting one of the imaging stages as a primary capture unit which is to be primarily used for capturing an image of the scene that is stored by the digital camera;

using the imaging stage not selected as the primary capture unit as a scene analysis capture unit for capturing scene data;

generating scene analysis data from the scene data for setting the capture parameters for the primary capture unit; and

capturing and displaying a preview image from the primary capture unit using the capture parameters generated from the scene analysis data.

18. The method as claimed in claim 17 further comprising the steps of:

capturing a further image from the scene analysis capture unit and generating new scene data;

determining from the new scene data if scene conditions have changed; and

resetting the capture parameters for the primary capture unit if the scene conditions have changed.

19. The method as claimed in claim 18 wherein the changed scene conditions are compared to a threshold and the capture parameters for the primary capture unit are reset if the change in scene conditions exceeds the threshold.

20. The method as claimed in claim 17 further comprising the steps of:

analyzing the scene using the preview image and the scene analysis data;

capturing and displaying a new preview image from the primary capture unit using capture parameters generated from the preview image and the scene analysis data;

capturing a further scene image from the scene analysis capture unit and generating new scene data;

determining from an analysis of the new scene data and the new preview image if scene conditions have changed; and

resetting the capture parameters for the primary capture unit if the scene conditions have changed.

21. A method for operating a digital camera having first and second imaging stages for forming separate images of a scene, wherein each imaging stage has a zoom lens, said method comprising the steps of:

selecting one of the imaging stages as a primary capture unit which is to be primarily used for capturing an image of the scene that is stored by the digital camera, wherein the primary capture unit has a predetermined zoom range;

using the imaging stage not selected as the primary capture unit as a scene analysis capture unit for capturing scene data;

requesting a zoom position for the digital camera;

determining whether the requested zoom position is within the predetermined zoom range of the primary capture unit; and

reversing the imaging units if the requested zoom position is not within the predetermined zoom range, whereby the primary capture unit becomes the scene analysis capture unit and the scene analysis capture unit becomes the primary capture unit.

22. A method for operating a digital camera having first and second imaging stages for forming separate images of a scene, wherein the camera includes a shutter button for initiating an image exposure, said method comprising the steps of:

selecting one of the imaging stages as a primary capture unit which is to be primarily used for capturing an image of the scene that is stored by the digital camera;

responsive to initial actuation of the shutter button, capturing an initial preview image from the primary capture unit;

using the imaging stage not selected as the primary capture unit as a scene analysis capture unit for capturing scene data;

generating scene analysis data from the initial preview image and the scene data for setting the capture parameters for the primary capture unit; and

capturing and displaying another preview image from the primary capture unit using the capture parameters generated from the scene analysis data.

23. The method as claimed in claim 22 further comprising the step of:

responsive to further actuation of the shutter button, capturing a primary image from the primary capture unit using the capture parameters generated from the preview image and the scene analysis data.

24. A method for operating a digital camera having first and second imaging stages for forming separate images of a scene, wherein the camera includes a shutter button for initiating an image exposure, said method comprising the steps of:

using one of the imaging stages as a primary capture unit for capturing an image of the scene that is stored by the digital camera, and using the other imaging stage as a scene analysis capture unit for capturing an analysis image;

responsive to initial actuation of the shutter button, capturing a preview image from the primary capture unit;

capturing an analysis image from the scene analysis capture unit;

generating scene analysis data from the preview image and the analysis image for setting the capture parameters for the primary capture unit;

using the scene analysis capture unit as a secondary capture unit to capture an augmentation image;

responsive to further actuation of the shutter button, capturing a primary image from the primary capture unit and capturing an augmentation image from the secondary capture unit, in each case using the capture parameters generated from the preview image and the scene analysis data; and

producing an enhanced image from the primary and augmentation images.

25. A method for operating a digital camera having first and second imaging stages for forming separate images of a scene, wherein the camera includes a shutter button for initiating an image exposure, said method comprising the steps of:

using one of the imaging stages as a primary capture unit for capturing a primary image of the scene, and using the other imaging stage as a secondary capture unit for capturing a secondary image that is used to modify the primary image;

responsive to actuation of the shutter button, capturing a primary image from the primary capture unit and capturing a secondary image from the secondary capture unit; and

producing an enhanced primary image from the primary and secondary images.

26. The method as claimed in claim 25 wherein the capture parameters include an exposure value and the secondary image is captured at a different exposure value than the primary image, whereby the enhanced primary image extends the dynamic range of the primary image.

27. The method as claimed in claim 25 wherein the capture parameters include a focus position value and the secondary image is captured at a different focus position than the primary image, whereby the enhanced primary image modifies the depth of field of the primary image.

28. The method as claimed in claim 25 wherein the exposure value and focus position value are user-adjustable capture parameters that are set by the user of the camera for a particular image exposure.

29. The method as claimed in claim 28 wherein the exposure value and focus position value are set by the user of the camera such that the primary and secondary images are captured as images at the same exposure value and focus position, whereby the enhanced image is a stereo image generated from the primary and secondary images.

30. The method as claimed in claim 25 wherein the secondary capture unit is further used to capture an analysis image for scene analysis, said method further comprising the step of:

obtaining scene analysis data from the analysis image; and

generating capture parameters for the primary and secondary capture units from the scene analysis data, whereby the capture parameters are used by the primary capture unit for capturing the primary image and by the secondary capture unit for capturing the augmentation image.

31. A method for operating a digital camera having first and second imaging stages for forming separate images of a scene, said method comprising the steps of:

using the image from the first imaging stage as a primary output image for forming a captured image signal;

using the image from the second imaging stage as a secondary output image having attributes useful for modifying the primary output image; and

augmenting the primary output image with the secondary output image, thereby generating an enhanced, captured image signal.

32. The method as claimed in claim 31 wherein the secondary output image is used to sharpen portions of the primary image.

33. The method as claimed in claim 32 wherein the secondary output image is used to extend the depth of field of the primary still image.

34. The method as claimed in claim 32 wherein the secondary output image is used to selectively sharpen portions of the primary still image and defocus other portions of the image.

35. The method as claimed in claim 31 wherein the secondary output image is captured by the second imaging stage at a different focus position than used by the first imaging stage to capture the primary output image.

36. The method as claimed in claim 31 wherein the secondary output image is used to modify the dynamic range of the primary image.

37. The method as claimed in claim 31 wherein the secondary output image is captured by the second imaging stage at a different exposure level than used by the first imaging stage to capture the primary output image.

38. The method as claimed in claim 31 wherein the captured image signal is a still image signal and the secondary output image modifies at least one of the depth of field and the dynamic range of the still image signal.

39. The method as claimed in claim 31 wherein the captured image signal is a video image signal and the secondary output image modifies at least one of the depth of field and the dynamic range of the video image signal.

40. The electronic camera as claimed in claim 31 wherein the imaging stages are set for different exposure times so that different levels of noise and motion blur are present in the respective primary and secondary output images, whereby a modified primary image is then formed by replacing portions of the primary image with portions of the secondary image such that the level of motion blur is reduced while minimizing an increase in the noise level.