CAMERA USING MULTIPLE LENSES AND IMAGE SENSORS IN A RANGEFINDER CONFIGURATION TO PROVIDE A RANGE MAP

• US 20080218612A1
• Filed: 03/09/2007
• Published: 09/11/2008
• Est. Priority Date: 03/09/2007
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Enge, Amy D., Pillman, Bruce H., Border, John N., Gallagher, Andrew C., Morales, Efrain O., Jacoby, Keith A., Palum, Russell J., Adams, James E.
• Granted Patent: US 7,683,962 B2
• Time in Patent Office: Days
• US Class Current: 348/262
• CPC Class Codes: G01S 19/45 by combining measurements of signals from the satellite radio beacon positioning system with a supplementary measurement
• CPC Class Codes: G03B 13/36 Autofocus systems
• CPC Class Codes: G03B 15/00 Special procedures for taking photographs; Apparatus therefor
• CPC Class Codes: G03B 29/00 Combinations of cameras, projectors or photographic printing apparatus with non-photographic non-optical apparatus, e.g. clocks or weapons; Cameras having the shape of other objects combinations with flash apparatus G03B15/03
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 5/2226 Determination of depth image, e.g. for foreground/background separation determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10

Abstract

An electronic camera for producing an output image of a scene from a captured image signal includes a first imaging stage comprising a first image sensor for generating a first sensor output and a first lens for forming a first image of the scene on the first image sensor, and a second imaging stage comprising a second image sensor for generating a second sensor output and a second lens for forming a second image of the scene on the second image sensor, where the lenses have different focal lengths. A processing stage uses the sensor output from one of the imaging stages as the captured image signal and uses the images from both imaging stages to generate a range map identifying distances to the different portions of the scene.

Claims

1. An electronic camera for producing an output image of a scene from a captured image signal, said electronic camera comprising:

a first imaging stage comprising a first image sensor for generating a first sensor output and a first lens for forming a first image of the scene on the first image sensor;

a second imaging stage comprising a second image sensor for generating a second sensor output and a second lens for forming a second image of the scene on the second image sensor, wherein the first and second lenses have different focal lengths; and

a processing stage for using the sensor output from one of the imaging stages as the captured image signal and using the images from both imaging stages to generate a range map identifying the distances to the different portions of the scene.

2. The electronic camera as claimed in claim 1 wherein the processing stage correlates the first and second images to determine a pixel offset between the images for different portions of the images, converts the pixel offsets into distances from the electronic camera, and produces the range map showing the distances to different portions of the images.

3. The electronic camera as claimed in claim 2 wherein the processing stage crops and upsamples the image from the lower focal length lens so that corresponding features in the two images span substantially the same number of pixels.

4. The electronic camera as claimed in claim 1 wherein the processing stage uses a rangefinder calibration curve to convert the pixel offsets into distances from the electronic camera.

5. The electronic camera as claimed in claim 1 wherein the camera further comprises a GPS unit for providing GPS coordinates for the location of the camera and an electronic compass for providing the pointing direction of the camera.

6. The electronic camera as claimed in claim 5 wherein the processing stage uses the GPS coordinates for the location of the camera, the pointing direction of the camera, and distance offsets from the range map in order to generate GPS coordinates for portions of the scene.

7. The electronic camera as claimed in claim 1 wherein the range map is used to modify the captured image.

8. An electronic camera for producing an output image of a scene from a captured image signal, said electronic camera comprising:

a first imaging stage comprising a first image sensor for generating a first sensor output;

a first lens for forming a first image of the scene on the first image sensor; and

a first lens focus adjuster for adjusting focus of the first lens responsive to a first focus detection signal;

a second imaging stage comprising a second image sensor for generating a second sensor output;

a second lens for forming a second image of the scene on the second image sensor; and

a second lens focus adjuster for adjusting focus of the second lens responsive to a second focus detection signal;

a processing stage for selecting either (a) the sensor output from the first imaging stage as the captured image signal and using the sensor output from at least the second imaging stage to generate the first focus detection signal for the selected imaging stage or (b) the sensor output from the second imaging stage as the captured image signal and using the sensor output from at least the first imaging stage to generate the second focus detection signal for the selected imaging stage, whereby the focus detection signal is applied to the lens focus adjuster of the selected imaging stage in order to adjust the focus of the image providing the sensor output for the captured image signal; and

wherein the processing stage further uses the sensor outputs from both the first and second imaging stages in a rangefinder configuration to produce a range map identifying distances to different portions of the scene.

9. The electronic camera as claimed in claim 8 wherein the camera further comprises a GPS unit for providing GPS coordinates for the location of the camera and an electronic compass for providing the pointing direction of the camera.

10. The electronic camera as claimed in claim 9 wherein the processing stage uses the GPS coordinates for the location of the camera, the pointing direction of the camera, and distance offsets from the range map in order to generate GPS coordinates for portions of the scene.

11. The electronic camera as claimed in claim 8 wherein the range map is used to modify the image.

12. An electronic camera for producing an output image of a scene from a captured image signal, said electronic camera comprising:

a first imaging stage comprising a first image sensor for generating a first sensor output;

a first lens for forming a first image of the scene on the first image sensor; and

a first lens focus adjuster for continually readjusting focus of the first lens responsive to a focus change detection signal;

a second imaging stage comprising a second image sensor for generating a second sensor output and a second lens for forming a second image of the scene on the second image sensor; and

a processing stage for using the sensor output from the first imaging stage as the captured image signal and using the sensor outputs from the first and second imaging stages in a rangefinder configuration to detect focus of the image of the scene and to generate the focus detection signal for the first imaging stage in order to focus the image, whereby the focus detection signal is applied to the lens focus adjuster of the first imaging stage in order to adjust the focus of the image providing the captured image signal; and

wherein the processing stage further uses the sensor outputs from both the first and second imaging stages in the rangefinder configuration to produce a range map identifying distances to different portions of the scene.

13. The electronic camera as claimed in claim 12 wherein the camera further comprises a GPS unit for providing GPS coordinates for the location of the camera and an electronic compass for providing the pointing direction of the camera.

14. The electronic camera as claimed in claim 13 wherein the processing stage uses the GPS coordinates for the location of the camera, the pointing direction of the camera, and distance offsets from the range map in order to generate GPS coordinates for portions of the scene.

15. The electronic camera as claimed in claim 12 wherein the range map is used to modify the captured image signal.

16. A method for using a digital camera, having two imaging stages, for producing an output image and a range map of a scene from a captured image signal, said method comprising the steps of:

forming a first image of the scene from a sensor output of a first image sensor located in a first imaging stage;

forming a second image of the scene from a sensor output of a second image sensor located in a second imaging stage, wherein the first and second images have different angles of view;

selecting the sensor output from one of the imaging stages as the captured image signal; and

using the images from both imaging stages to generate a range map identifying the distances to different portions of the scene.

17. The method as claimed in claim 16 further comprising the steps of:

correlating the first and second images to determine a pixel offset between the images for different portions of the images;

converting the pixel offsets into distances from the electronic camera; and

producing the range map showing the distances to different portions of the images.

18. The method as claimed in claim 17 further comprising the step of:

cropping and upsampling the image from the wider angle image so that corresponding features in the two images span substantially the same number of pixels.

19. The method as claimed in claim 16 wherein the range map is used to improve object identification within the output image by identifying continuous boundaries of an object so that the shape of the object can be defined.

20. The method as claimed in claim 16 wherein the range map is used to enable object extraction from an image by identifying continuous boundaries of an object so that it can be segmented within the output image.

21. The method as claimed in claim 16 wherein the range map is used to enable motion tracking of objects within multiple output images by identifying objects so that they can be tracked as the same object between the output images.

22. The method as claimed in claim 16 wherein the range map is used to enable dynamic depth of field images by blurring of portions of the output image that correspond to areas of the scene that lie outside of a desired depth of field.

23. The method as claimed in claim 16 wherein the range map is used to improve dynamic range within the output image by applying gain adjustments to objects as a whole within the output image.

24. The method as claimed in claim 16 wherein the range map is used to reduce exposure issues introduced by use of a flash by reducing gain on portions of the output image that correspond to objects in a foreground area and increasing the gain on objects in a background area.

25. The method as claimed in claim 16 wherein the range map is used to improve scene balance within the output image by enabling objects in a foreground area to be emphasized.