PERSPECTIVE CORRECTION USING A REFLECTION

• US 20130137482A1
• Filed: 11/30/2011
• Published: 05/30/2013
• Est. Priority Date: 11/30/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Brown, Shawn M., Cosman, Stephen, Pengelly, Robert, Sheikh, Suhel
• Granted Patent: US 8,774,556 B2
• Time in Patent Office: Days
• US Class Current: 455/556.1
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20172 Image enhancement details
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 1/00244 with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324

Abstract

Techniques and tools are described for performing perspective correction using a reflection. Reflective properties of a surface being photographed can be used to determine a rotation of the device taking the photograph relative to the surface. Light sourced or produced by the device can be used to create a reflection spot in the picture. A position of the reflection spot within the picture is calculated and used to determine the rotation. The rotation can be used for performing perspective correction on the picture, or on another picture taken by the device.

Claims

1. A method for performing perspective correction, comprising:

receiving image data corresponding to a picture captured by a picture-taking device, the picture having a reflection spot caused by reflection of light sourced by the picture-taking device off a surface;

calculating a position of the reflection spot within the picture from the image data; and

determining a rotation of the picture-taking device relative to the surface using the calculated position, the rotation being used for performing the perspective correction.

2. The method of claim 1, wherein the calculating of the position of the reflection spot within the picture comprises determining an x coordinate and a y coordinate for the reflection spot from the image data, and the determining of the rotation of the picture-taking device relative to the surface uses the x coordinate and the y coordinate.

3. The method of claim 1, wherein the reflection spot corresponds to a region of saturation within the picture, the calculating of the position of the reflection spot comprises determining an x coordinate and a y coordinate for a center of the region of saturation from the image data, and the determining of the rotation of the picture-taking device relative to the surface uses the x coordinate and the y coordinate.

4. The method of claim 1, wherein the reflection spot corresponds to a region of saturation within the picture, the region defined by a plurality of saturated pixels, and the calculating of the position of the reflection spot comprises determining a distance in pixels from a center of the picture for the plurality in saturated pixels.

5. The method of claim 1, wherein the surface is a vertical surface and the determining of the rotation of the picture-taking device comprises determining a vertical tilt angle and a horizontal tilt angle of the picture-taking device.

6. The method of claim 1, further comprising:

applying a perspective correction transform to the image data using the rotation of the picture-taking device.

7. The method of claim 1, wherein the image data is a first set of image data that corresponds to a first picture of a viewing area, the surface being in the viewing area, further comprising:

receiving a second set of image data corresponding to a second picture of the surface and the viewing area; and

applying a perspective correction transform to the second image using the rotation of the picture-taking device.

8. The method of claim 1, further comprising:

receiving one or more parameters of the picture-taking device related to an angle of view of the picture-taking device, wherein the determining of the rotation of the picture-taking device is performed using the position of the reflection spot within the picture and the one or more parameters.

9. The method of claim 1, wherein the picture-taking device is a camera that is part of a mobile telephone.

10. The method of claim 1, wherein the calculating of the position of the reflection spot within the picture comprises determining an x coordinate along an x axis and a y coordinate along a y axis for the reflection spot from the image data, the method further comprising:

generating the light sourced by the picture-taking device;

capturing the picture having the reflection spot;

determining a first width along the x-axis and a second width along the y-axis for the picture from the image data;

receiving a predetermined value for an angle of view of the picture-taking device, wherein the determining of the rotation of the picture-taking device relative to the surface is performed using the angle of view, the first and the second width, the x coordinate, and the y coordinate; and

applying a perspective correction transform to the image using the rotation of the picture-taking device.

11. The method of claim 1, wherein the surface is a surface of a whiteboard and the picture does not include an edge of the whiteboard, or the picture includes the edge of the whiteboard and the method further comprises applying a perspective correction transform to the image data using the rotation of the picture-taking device and without using a position of the edge of the whiteboard within the picture.

12. One or more computer storage media storing computer-executable instructions, which, when executed by a computer, cause the computer to perform a method comprising:

receiving an image taken by a picture-taking device of a portion of a surface, the image having a plurality of pixels corresponding to a reflection off of the surface of light produced by the picture-taking device;

calculating a position within the image for the plurality of pixels; and

determining a tilt angle of the picture-taking device using the calculated position of the plurality of pixels within the image.

13. The one or more computer storage media of claim 12, wherein the method further comprises:

receiving or calculating a value for an angle of view of the picture-taking device, wherein the determining of the tilt angle of the picture-taking device is performed using the calculated position for the plurality of pixels and the value for the angle of view.

14. The one or more computer storage media of claim 12, wherein the calculating of the position for the plurality of pixels comprises calculating the position within the image for a plurality of saturated pixels.

15. The one or more computer storage media of claim 12, wherein the calculating of the position within the image for the plurality of pixels comprises calculating an x coordinate and a y coordinate for a center of the plurality of pixels.

16. The one or more computer storage media of claim 12, wherein the calculating of the position within the image for the plurality of pixels comprises calculating a distance (DX) along an x-axis and a distance (DY) along a y-axis for a center of the plurality of pixels, the method further comprising:

calculating a width (WX) along the x-axis and a width (WY) along the y-axis for the picture; and

receiving or calculating values for an angle of view (AOVX) along the x-axis and an angle of view (AOVY) along the y-axis of the picture-taking device, wherein the determining of the rotation of the picture-taking device relative to the surface comprises calculating a tilt angle along the x-axis using arctan((DX/WX)*tan(AOVX/2)) and calculating a tilt angle along the y-axis using arctan((DY/WY)*tan(AOVY/2)).

17. The one or more computer storage media of claim 12, wherein the method further comprises:

applying perspective correction techniques to the image using the tilt angle, or receiving a second image of the portion of the surface and applying perspective correction techniques to the second image using the tilt angle.

18. A mobile telephone, comprising:

a camera characterized by an angle of view and configured to generate images of an area within the angle of view;

a flash unit configured to produce a flash of light to illuminate the area, the camera being configured to generate the images while the flash of light is illuminating the area; and

an image processing system configured to receive the images, and, for images having a reflection spot produced by the flash of light reflecting off a surface in the area, to determine an x coordinate and a y coordinate of the reflection spot within the image and to determine the tilt angle for the camera based on the x coordinate, y coordinate, and the angle of view; and

a perspective correction system configured to apply a perspective correction transform to the at least one of the received images using the tilt angle.

19. The mobile telephone of claim 18, wherein the flash unit is configured to produce a flash of infrared light.

20. The mobile telephone of claim 18, wherein the images are pre-images of the area within the angle of view, and the camera is further configured to generate an additional image of the area while the flash unit is not producing the flash of light, the perspective correction transform being applied to the additional image.