Bayesian image super resolution

• US 20040170340A1
• Filed: 02/27/2003
• Published: 09/02/2004
• Est. Priority Date: 02/27/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Tipping, Michael E., Bishop, Christopher M.
• Granted Patent: US 7,106,914 B2
• Time in Patent Office: Days
• US Class Current: 382/299
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 7/35 using statistical methods

Abstract

An image super resolution system computes a high resolution image of a target from multiple low resolution images of the same target. Each low resolution image differs slightly in perspective from each of the other low resolution images. A coarse registration operation determines initial estimates of registration parameters (e.g., representing shifts and rotation in perspective) associated with each low resolution image. A fine registration operation improves the initial estimates using Bayesian analysis to infer the registration parameters and an acuity parameter. As such, a marginal likelihood of the low resolution images is optimized to determine the improved estimates of the registration parameters and the acuity parameter, which are used to solve for the high resolution image.

Claims

1. A method of computing a high resolution image of a target from multiple low resolution images taken of the target by an image capture device, each low resolution image having at least one difference in perspective relative to the target as compared to each other low resolution image, each difference in perspective being represented by a registration parameter, the method comprising:

optimizing a marginal likelihood associated with the low resolution images that is dependent upon one or more registration parameters of the low resolution images and an acuity parameter associated with the image capture device and is independent of the high resolution image to determine estimates of the one or more registration parameters and the acuity parameter;

computing a mean of a posterior distribution associated with the high resolution image using the estimates of the one or more registration parameters and the acuity parameter to determine the high resolution image.

2. The method of claim 1 wherein the at least one difference of perspective results from a shift of the image capture device.

3. The method of claim 1 wherein the at least one difference of perspective results from a rotation of the image capture device.

4. The method of claim 1 wherein the marginal likelihood is determined by a Bayesian technique that integrates out dependence of the marginal likelihood on the high resolution image.

5. The method of claim 1 further comprising:

determining initial estimates using coarse registration of the low resolution images to initialize the optimizing operation.

6. A computer program product encoding a computer program for executing on a computer system a computer process for computing a high resolution image of a target from multiple low resolution images taken of the target by an image capture device, each low resolution image having at least one difference in perspective relative to the target as compared to each other low resolution image, each difference in perspective being represented by a registration parameter, the computer process comprising:

optimizing a marginal likelihood associated with the low resolution images that is dependent upon one or more registration parameters of the low resolution images and an acuity parameter associated with the image capture device and is independent of the high resolution image to determine estimates of the one or more registration parameters and the acuity parameter;

computing a mean of a posterior distribution associated with the high resolution image using the estimates of the one or more registration parameters and the acuity parameter to determine the high resolution image.

7. The computer program product of claim 6 wherein the at least one difference of perspective results from a shift of the image capture device.

8. The computer program product of claim 6 wherein the at least one difference of perspective results from a rotation of the image capture device.

9. The computer program product of claim 6 wherein the marginal likelihood is determined by a Bayesian technique that integrates out dependence of the marginal likelihood on the high resolution image.

10. The computer program product of claim 6 wherein the computer process further comprises:

determining initial estimates using coarse registration of the low resolution images to initialize the optimizing operation.

11. A system for computing a high resolution image of a target from multiple low resolution images taken of the target by an image capture device, each low resolution image having at least one difference in perspective relative to the target as compared to each other low resolution image, each difference in perspective being represented by a registration parameter, the system comprising:

a fine registration module optimizing a marginal likelihood associated with the low resolution images that is dependent upon one or more registration parameters of the low resolution images and an acuity parameter associated with the image capture device and is independent of the high resolution image to determine estimates of the one or more registration parameters and the acuity parameter;

a high resolution inference module computing a mean of a posterior distribution associated with the high resolution image using the estimates of the one or more registration parameters and the acuity parameter to determine the high resolution image.

12. The system of claim 11 wherein the at least one difference of perspective results from a shift of the image capture device.

13. The system of claim 11 wherein the at least one difference of perspective results from a rotation of the image capture device.

14. The system of claim 11 wherein the marginal likelihood is determined by a Bayesian technique that integrates out dependence of the marginal likelihood on the high resolution image.

15. The system of claim 11 further comprising:

a coarse registration module determining initial estimates using coarse registration of the low resolution images to initialize the optimizing operation.