Depth of field enhancement

• US 6,320,979 B1
• Filed: 10/06/1998
• Issued: 11/20/2001
• Est. Priority Date: 10/06/1998
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Melen, Roger D.
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Werner, Brian P.
• Application Number:
  US09/167,641
• Time in Patent Office: 1,141 Days
• Field of Search: 382/154, 382/148, 382/151, 382/173, 382/255, 382/275, 382/294, 382/299, 382/318, 348/50, 359/368, 359/377, 359/384, 359/381, 359/376, 359/378, 359/372, 359/369
• US Class Current: 382/154
• CPC Class Codes: G02B 21/367 providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
• CPC Class Codes: G02B 27/0075 with means for altering, e.g. increasing, the depth of field or depth of focus
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method

Abstract

A lens system (100) and image plane (104) are used to capture a number of sample images (400) of a three dimensional scene, each sample image (400) corresponding to a different object distance (106). The in-focus portions of each of the sample images (400) are then merged into a composite image (620) which appears to have a depth of field greater than any of the sample images (400). In one embodiment, the lens system (100) is movably attached to a camera housing (302) such that a motor (304) can move the lens system (100) in the direction of the central axis of the lens system (100). A two dimensional array (300) of photo-sensors is mounted in the camera along the central axis of the lens system (100). As the lens system (100) is traversed along the central axis, points at various distances in front of the lens system (100) pass in and out of focus on the array (300) of photo-sensors. Image information is captured from the array (300) as it is traversed, producing a series of depth-differentiated sample images (400). The in-focus portion of each sample image (400) is identified through contrast analysis. The in-focus portions identified are added to produce a composite image (620) of the scene exhibiting an apparently large depth of field.

Claims

1. A method for using an imaging system and an array of sensor locations to construct a composite image of a three dimensional scene, the array of sensor locations defining an image plane on which is focused an image of points a focus distance in front of the image plane, the method comprising the steps of:

moving the imaging system relative to the image plane so as to change the focus distance;

at each sensor location of the array, for at least two distinct focus distances;

sensing radiation which has passed through the imaging system;

producing image information based on the sensed radiation including a brightest pixel value and a darkest pixel value;

calculating a contrast value as a ratio of the brightest pixel value within the sensor location to the darkest pixel value within the sensor location; and

assigning the calculated contrast value to a plurality of pixels within the sensor location;

recording the image information; and

constructing a composite image by incorporating into said composite image, for each sensor location, image information from that focus distance which produces the maximum contrast.

2. The method of claim 1, wherein the imaging system comprises one of a lens system and a mirror system.

3. The method of claim 1, wherein the imaging system comprises a convex lens.

4. The method of claim 1, wherein the radiation is sensed at each sensing location by one of a Charge Coupled Device (CCD) cell, a photosensitive film region, and an X-ray film region.

5. The method of claim 1, wherein the radiation is one of visible light and X-ray radiation.

6. The method of claim 1 wherein the radiation is sensed by photosensitive film, and a separate frame of film is exposed for each position of the imaging system relative to the image plane.

7. The method of claim 1 wherein each sensor location contains at least one pixel that is also contained within at least one other sensor location.

8. A method for using at least two sample images of a three dimensional scene to construct a composite image of the scene, each of the sample images corresponding to a unique focus distance, the method comprising the steps of:

for each sample image and for each of at least two measurement regions, each measurement region corresponding to a portion of the scene represented in the sample images, determining contrast information as a ratio of a brightest pixel value within each measurement region to a darkest pixel value within each measurement region, and assigning the contrast value to a plurality of pixels within each measurement region; and

combining image information from all of the sample images, on a measurement region by measurement region basis, based upon which sample image exhibits the maximum contrast for that measurement region, to construct the composite image.

9. The method of claim 8, wherein the sample images are digital images.

10. The method of claim 8, wherein the sample images are photographic images.

11. The method of claim 8, wherein the method is a computer-implemented method.

12. A computer-readable medium containing a computer program for using at least two sample images of a three dimensional scene to construct a composite image of the scene, each of the sample images corresponding to a unique focus distance, the computer program comprising an array of instructions for causing a central processing unit (CPU) to perform the steps of:

for each sample image and for each of at least two measurement regions, each measurement region corresponding to a portion of the scene represented in the sample images, determining contrast information as a ratio of a brightest pixel value within each measurement region to a darkest pixel value within each measurement region, and assigning the contrast value to a plurality of pixels within each measurement region; and

combining image information from all of the sample images, on a measurement region by measurement region basis, based upon which sample image exhibits the maximum contrast for that measurement region, to construct the composite image.

13. The computer-readable medium of claim 12, wherein the sample images are digital images.

14. The computer-readable medium of claim 12, wherein the sample images are photographic images.

15. A system for constructing a composite image of a three dimensional scene, the system comprising:

an array of sensor locations defining an image plane, each sensor location being adapted for sensing radiation, and for producing radiation information including a brightest pixel value and a darkest pixel value;

an imaging system for focusing on the image plane an image of points a focus distance away from the image plane, the imaging system being movably coupled to the array such that the array is mounted along an d generally orthogonal to a central axis of the imaging system;

a motor coupled to the imaging system and to the array for, in response to a command signal, changing the distance between the imaging system and the array, such as to change the focus distance; and

a processor module coupled to the array and to the motor, for;

sending command signals to the motor to cause it to change the focus distance;

receiving radiation information from each of the sensor locations; and

constructing a composite image by calculating a contrast value as a ratio of the brightest pixel value within the sensor location to the darkest pixel value within the sensor location, assigning the calculated contrast value to a plurality of pixels within the sensor location, and incorporating into said composite image, for each sensor location, radiation information from that focus distance which produces the maximum contrast.

16. The system of claim 15, wherein the imaging system comprises one of a lens system and a mirror system.

17. The system of claim 15, wherein the radiation is sensed at each sensing location by one of a Charge Coupled Device (CCD) cell, a photosensitive film region, and an X-ray film region.

18. The system of claim 15, wherein the radiation is one of visible light and X-ray radiation.