Adaptive Image Acquisition for Multiframe Reconstruction

• US 20150264245A1
• Filed: 11/24/2014
• Published: 09/17/2015
• Est. Priority Date: 03/26/2008
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Robinson, M. Dirk, Yang, Junlan, Stork, David G.
• Granted Patent: US 9,438,816 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/743 Bracketing, i.e. taking a series of images with varying exposure conditions

Abstract

Multiframe reconstruction combines a set of acquired images into a reconstructed image. Here, which images to acquire are selected based at least in part on the content of previously acquired images. In one approach, a set of at least three images of an object are acquired at different acquisition settings. For at least one of the images in the set, the acquisition setting for the image is determined based at least in part on the content of previously acquired images. Multiframe image reconstruction is applied to the set of acquired images to synthesize a reconstructed image of the object.

Claims

1. A method for multiframe reconstruction of an object comprising:

acquiring a set of at least two images of the object at different acquisition settings;

for at least one of the images, determining the acquisition setting for the image based at least in part on content of previously acquired images; and

applying multiframe reconstruction to the set of images.

2. The method of claim 1 wherein the steps of acquiring a set of images and determining an acquisition setting comprise:

acquiring at least two initial images at acquisition settings that do not depend on content of previously acquired images; and

for every image acquired after the initial images, determining the acquisition setting for the image based at least in part on content of previously acquired images.

3. The method of claim 2 wherein, for every image acquired after the initial images, the step of determining the acquisition setting for said image does not depend on whether any additional images will be acquired after said image.

4. The method of claim 2 wherein, for at least one of the images acquired after the initial images, the step of determining the acquisition setting for said image is based at least in part on assuming that at least one additional image will be acquired after said image.

5. The method of claim 2 wherein the set of images contains K images;

and, for every image acquired after the initial images, the step of determining the acquisition setting for said image is based at least in part on assuming a final set containing K images.

6. The method of claim 1 wherein the step of determining the acquisition setting for the image based at least in part on content of previously acquired images comprises:

determining the acquisition setting for said image based at least in part on increasing an image quality performance of the multiframe reconstruction, given the previously acquired images.

7. The method of claim 6 wherein the step of determining the acquisition setting for said image based at least in part on increasing an effective signal-to-noise ratio.

8. The method of claim 6 wherein the step of determining the acquisition setting for said image based at least in part on increasing a performance of the multiframe reconstruction is further based on the predicted MSE performance.

9. The method of claim 6 wherein the step of determining the acquisition setting for said image based at least in part on increasing a performance of the multiframe reconstruction comprises determining the acquisition setting based at least in part on increasing information captured by said image, compared to information captured by previously acquired images.

10. The method of claim 9 wherein said information is a measure of a predicted sharpness of the multiframe reconstructed image.

11. The method of claim 9 wherein said information is a measure of a change in local smoothness related to object depth.

12. The method of claim 6 wherein the step of determining the acquisition setting for the image based at least in part on content of previously acquired images further comprises:

determining the acquisition setting for said image based at least in part on reducing a total power consumption.

13. The method of claim 6 wherein the step of determining the acquisition setting for said image based at least in part on increasing a performance of the multiframe reconstruction is based on modeling a relationship between the acquisition settings and an optical point spread function.

14. The method of claim 1 where the step of determining the acquisition setting for the image is based on maximizing a weighted sum of total sharpness over a collection of image regions in the previously acquired images.

15. The method of claim 1 wherein the acquisition setting includes an aperture setting.

16. The method of claim 1 wherein the acquisition setting includes a focus setting.

17. The method of claim 1 wherein the acquisition setting includes an exposure setting.

18. The method of claim 1 wherein the object is modeled as a multi-depth object.

19. The method of claim 18 wherein the step of determining the acquisition setting comprises:

estimating different object depths for different tiles of the multi-depth object, based at least in part on content of previously acquired images; and

determining the acquisition setting for the image based at least in part on the estimated object depths.

20. The method of claim 19 wherein the step of determining the acquisition setting for the image is based at least in part on a weighted average over the tiles of a merit function evaluated for each tile.