Automatic Adjustment of Images using a Homography

• US 20130287318A1
• Filed: 04/26/2013
• Published: 10/31/2013
• Est. Priority Date: 04/27/2012
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Shechtman, Elya, Wang, Jue, Lee, Hyunjoon, Lee, Seungyong
• Granted Patent: US 9,008,460 B2
• Time in Patent Office: Days
• US Class Current: 382/289
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 3/608 by skew deformation, e.g. two-pass or three-pass rotation
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

Techniques and apparatus for automatic upright adjustment of digital images. An automatic upright adjustment technique is described that may provide an automated approach for straightening up slanted features in an input image to improve its perceptual quality. This correction may be referred to as upright adjustment. A set of criteria based on human perception may be used in the upright adjustment. A reprojection technique that implements an optimization framework is described that yields an optimal homography for adjustment based on the criteria and adjusts the image according to new camera parameters generated by the optimization. An optimization-based camera calibration technique is described that simultaneously estimates vanishing lines and points as well as camera parameters for an image; the calibration technique may, for example, be used to generate estimates of camera parameters and vanishing points and lines that are input to the reprojection technique.

Claims

1. A method implemented by one or more computing devices, the method comprising:

obtaining camera information for an image of a scene that includes one or more features having lines;

computing a homography based on the camera information that is to be applied to the image to adjust alignment of the one or more features of the scene in relation to an axis of the image, the homography computed at least in part according to one or more human perception-based criteria; and

reprojecting the image according to the homography to generate an adjusted image of the scene having the adjusted alignment of the lines of the one or more features of the scene with the axis of the image.

2. A method as described in claim 1, wherein the axis is a vertical or horizontal axis and the one or more features are substantially aligned with the vertical or horizontal axis in the adjusted image.

3. A method as described in claim 1, wherein the homography describes new camera information according to the one or more human perception-based criteria.

4. A method as described in claim 1, wherein the camera information includes an intrinsic camera parameter matrix and three dimensional rotation matrices.

5. A method as described in claim 4, wherein the camera information further includes vanishing point information.

6. A method as described in claim 1, wherein the homography includes a new intrinsic camera parameter matrix and new three dimensional rotation matrices.

7. A method as described in claim 1, wherein the homography includes a translation.

8. A method as described in claim 1, wherein the obtaining includes applying a camera calibration technique to estimate the camera information for the image.

9. A method as described in claim 8, wherein the camera calibration technique is optimization-based to estimate intrinsic camera parameters, a three dimensional rotation, and vanishing point information in conjunction with each other for the image of the scene.

10. A method as described in claim 1, wherein the one or more human perception-based criteria include a picture frame alignment criterion that is usable to constrain alignment of the one or more features according to a picture frame defined by the image.

11. A method as described in claim 1, wherein the one or more human perception-based criteria include an eye-level alignment criterion that is usable to constrain alignment of the one or more features according to a line connecting two vanishing points of the scene.

12. A method as described in claim 1, wherein the one or more human perception-based criteria include an image distortion criterion that is usable to constrain distortion of the one or more features.

13. A method as described in claim 1, wherein the homography includes focal lengths in vertical and horizontal directions, the focal length in each direction being allowed to vary, one to another.

14. One or more computer-readable storage media comprising instructions stored thereon that, responsive to execution by a computing device, causes the computing device to perform operations comprising:

computing a homography based on camera information for an image of a scene that includes one or more features having lines, the homography computed at least in part according to one or more human perception-based criteria; and

generating an adjusted image of the scene according to the homography, the adjusted image having an alignment of the lines of the one or more features of the scene that is adjusted in relation to an axis of the image.

15. One or more computer-readable storage media as described in claim 14, wherein the axis is a vertical or horizontal axis and the one or more features are substantially aligned with the vertical or horizontal axis in the adjusted image.

16. One or more computer-readable storage media as described in claim 14, wherein the homography describes new camera information according to the one or more human perception-based criteria.

17. One or more computer-readable storage media as described in claim 14, wherein:

the camera information includes an intrinsic camera parameter matrix and three dimensional rotation matrices; and

the homography includes a new intrinsic camera parameter matrix and new three dimensional rotation matrices.

18. One or more computer-readable storage media as described in claim 14, wherein the homography includes a translation.

19. A system comprising:

one or more computing devices configured to perform operations including;

computing a homography based on camera information for an image of a scene that includes one or more features having lines, the homography computed at least in part according to one or more human perception-based criteria; and

generating an adjusted image of the scene according to the homography, the adjusted image having an alignment of the lines of the one or more features of the scene that is adjusted in relation to an axis of the image.

20. A system as described in claim 19, further comprising applying a camera calibration technique to estimate the camera information for the image.