Metadata-driven method and apparatus for constraining solution space in image processing techniques

• US 10,068,317 B2
• Filed: 11/21/2012
• Issued: 09/04/2018
• Est. Priority Date: 08/29/2008
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Chen, Simon, Chien, Jen-Chan, Jin, Hailin
• Primary Examiner(s): Bali, Vikkram
• Application Number:
  US13/683,166
• Publication Number:
  US 20130089262A1
• Time in Patent Office: 2,113 Days
• Field of Search: None
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 1/32101 Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
• CPC Class Codes: H04N 2201/3252 Image capture parameters, e.g. resolution, illumination conditions, orientation of the image capture device H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise

Abstract

Methods and apparatus for constraining solution space in image processing techniques may use the metadata for a set of images to constrain an image processing solution to a smaller solution space. In one embodiment, a process may require N parameters for processing an image. A determination may be made from metadata that multiple images were captured with the same camera/lens and with the same settings. A set of values may be estimated for the N parameters from data in one or more of the images. The process may then be applied to each of images using the set of values. In one embodiment, a value for a parameter of a process may be estimated for an image. If the estimated value deviates substantially from a value for the parameter in the metadata, the metadata value is used in the process instead of the estimated value.

Claims

1. A computer-implemented method, comprising:

performing, by one or more computing devices;

estimating, from image content of a captured image, a value for a parameter to be used in a digital image processing technique when applied to the captured image;

obtaining, from metadata corresponding to the captured image, a value for the parameter determined when capturing the captured image;

determining that a difference between the estimated value for the parameter and the value for the parameter obtained from the metadata exceeds a threshold; and

applying the digital image processing technique to the captured image to generate an output image, the value for the parameter obtained from the metadata used in the digital image processing technique instead of the estimated value in response to said determining that the difference exceeds the threshold.

2. The computer-implemented method as recited in claim 1, wherein the digital image processing technique is a vignette removal process, and wherein the parameter is exposure.

3. The computer-implemented method as recited in claim 1, wherein the digital image processing technique is a vignette removal process, and wherein the parameter is one of one or more parameters used in a vignette model applied during the vignette removal process.

4. The computer-implemented method as recited in claim 1, wherein the digital image processing technique is a lens distortion removal process, and wherein the parameter is one of one or more parameters used in a lens distortion model applied during the lens distortion removal process.

5. The computer-implemented method as recited in claim 1, wherein the digital image processing technique is a chromatic aberration removal process, and wherein the parameter is one of one or more parameters used in a chromatic aberration model applied during the chromatic aberration removal process.

6. The computer-implemented method as recited in claim 1, wherein the digital image processing technique is a sensor noise removal process, and wherein the parameter is one of one or more parameters used in a sensor noise model applied during the sensor noise removal process.

7. The computer-implemented method as recited in claim 1, further comprising, prior to said estimating, determining that the digital image processing technique is to be applied to the captured image according to the metadata corresponding to the images.

8. A system, comprising:

at least one processor; and

a memory comprising program instructions that are executable by the at least one processor to;

estimate, from image content of a captured image, a value for a parameter to be used in a digital image processing technique when applied to the captured image;

obtain, from metadata corresponding to the captured image, a value for the parameter determined when capturing the captured image;

determine that a difference between the estimated value for the parameter and the value for the parameter obtained from the metadata exceeds a threshold; and

apply the digital image processing technique to the captured image to generate an output image, the value for the parameter obtained from the metadata used in the digital image processing technique instead of the estimated value in response to a determination that the difference exceeds the threshold.

9. The system as recited in claim 8, wherein the digital image processing technique is a vignette removal process, and wherein the parameter is exposure.

10. The system as recited in claim 8, wherein the digital image processing technique is a vignette removal process, and wherein the parameter is one of one or more parameters used in a vignette model applied during the vignette removal process.

11. The system as recited in claim 8, wherein the digital image processing technique is a lens distortion removal process, and wherein the parameter is one of one or more parameters used in a lens distortion model applied during the lens distortion removal process.

12. The system as recited in claim 8, wherein the digital image processing technique is a chromatic aberration removal process, and wherein the parameter is one of one or more parameters used in a chromatic aberration model applied during the chromatic aberration removal process.

13. The system as recited in claim 8, wherein the digital image processing technique is a sensor noise removal process, and wherein the parameter is one of one or more parameters used in a sensor noise model applied during the sensor noise removal process.

14. A non-transitory computer-readable storage medium storing program instructions that are computer-executable by a processor of a computer device to implement operations comprising:

estimating, from image content of a captured image, a value for a parameter to be used in a digital image processing technique when applied to the captured image;

obtaining, from metadata corresponding to the captured image, a value for the parameter determined when capturing the captured image;

determining that a difference between the estimated value for the parameter and the value for the parameter obtained from the metadata exceeds a threshold; and

applying the digital image processing technique to the captured image to generate an output image, the value for the parameter obtained from the metadata used in the digital image processing technique instead of the estimated value in response to said determining that the difference exceeds the threshold.

15. The non-transitory computer-readable storage medium as recited in claim 14, wherein the digital image processing technique is a vignette removal process, and wherein the parameter is exposure.

16. The non-transitory computer-readable storage medium as recited in claim 14, wherein the digital image processing technique is a vignette removal process, and wherein the parameter is one of one or more parameters used in a vignette model applied during the vignette removal process.

17. The non-transitory computer-readable storage medium as recited in claim 14, wherein the digital image processing technique is a lens distortion removal process, and wherein the parameter is one of one or more parameters used in a lens distortion model applied during the lens distortion removal process.

18. The non-transitory computer-readable storage medium as recited in claim 14, wherein the digital image processing technique is a chromatic aberration removal process, and wherein the parameter is one of one or more parameters used in a chromatic aberration model applied during the chromatic aberration removal process.

19. The non-transitory computer-readable storage medium as recited in claim 14, wherein the digital image processing technique is a sensor noise removal process, and wherein the parameter is one of one or more parameters used in a sensor noise model applied during the sensor noise removal process.

20. The non-transitory computer-readable storage medium as recited in claim 14, wherein the program instructions are further computer-executable to implement, prior to said estimating, determining that the digital image processing technique is to be applied to the captured image according to the metadata corresponding to the images.