Perspective correction using a reflection

• US 8,983,227 B2
• Filed: 06/11/2014
• Issued: 03/17/2015
• Est. Priority Date: 11/30/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Sheikh, Suhel, Brown, Shawn M., Cosman, Stephen, Pengelly, Robert
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US14/302,278
• Publication Number:
  US 20140354831A1
• Time in Patent Office: 279 Days
• Field of Search: 382/260, 382/274, 382/275, 358/3.26, 358/3.27, 358/463
• US Class Current: 382/275
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20172 Image enhancement details
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 1/00244 with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324

Abstract

Techniques and tools are described for performing perspective correction using a reflection. Reflective properties of a surface being photographed can be used to determine a rotation of the device taking the photograph relative to the surface. Light sourced or produced by the device can be used to create a reflection spot in the picture. A position of the reflection spot within the picture is calculated and used to determine the rotation. The rotation can be used for performing perspective correction on the picture, or on another picture taken by the device.

Claims

1. A method for performing perspective correction, comprising:

receiving image data corresponding to one or more pictures captured by a picture-taking device, at least one of the pictures having a reflection spot caused by reflection of light sourced by the picture-taking device off a surface; and

applying a perspective correction transform to the image data using a rotation of the picture-taking device relative to the surface thereby generating corrected image data, wherein the rotation is based on a position of the reflection spot within the at least one of the pictures.

2. The method of claim 1, wherein the method is performed by a cloud-based service implemented by one or more computing devices, wherein the image data is received by the cloud-based service and wherein the applying is performed by the cloud-based service.

3. The method of claim 1, further comprising:

transmitting the corrected image data to the picture-taking device.

4. The method of claim 1, further comprising:

calculating the position of the reflection spot within the at least one of the pictures using the image data.

5. The method of claim 4, further comprising:

determining the rotation of the picture-taking device relative to the surface using the calculated position.

6. The method of claim 5, further comprising:

receiving one or more parameters of the picture-taking device related to an angle of view of the picture-taking device, wherein the determining of the rotation of the picture-taking device is performed using the position of the reflection spot within the at least one of the pictures and the one or more parameters.

7. The method of claim 1, further comprising:

receiving the position of the reflection spot within the at least one of the pictures or receiving the rotation of the picture-taking device relative to the surface.

8. The method of claim 1, further comprising:

storing at least one of the corrected image data and the rotation of the picture-taking device.

9. The method of claim 1, wherein:

the at least one of the pictures corresponds to a first picture of a viewing area, the surface being in the viewing area;

the one or more pictures includes a second picture of the surface and the viewing area; and

the perspective correction transform is applied to the second picture.

10. One or more server computers, comprising:

a connection to a network;

an image processing system configured to receive via the network images from a picture-taking device that is characterized by an angle of view, and, for images having a reflection spot, to determine a position of the reflection spot within the image and to determine a tilt angle for the picture-taking device based on the position and the angle of view of the picture-taking device; and

a perspective correction system configured to apply a perspective correction transform to the at least one of the received images using the tilt angle.

11. The one or more server computers of claim 10, wherein the perspective correction system is configured to transmit perspective corrected images to the picture-taking device.

12. The one or more server computers of claim 10, wherein the image processing system is further configured to receive the angle of view from the picture-taking device via the network.

13. The one or more server computers of claim 10, wherein the server computer operates a cloud-based service that provides perspective correction for the picture-taking device.

14. The one or more server computers of claim 10, wherein the image processing system is further configured to determine an x coordinate and a y coordinate for the reflection spot and the determining of the tilt angle for the picture-taking device uses the x coordinate and the y coordinate.

15. One or more computer-readable storage devices storing computer-executable instructions for causing one or more computing devices to perform operations providing a cloud-based service for perspective correction, the operations comprising:

receiving a position of a reflection spot within a picture of a surface in a viewing area, the reflection spot caused by reflection of light sourced by a picture-taking device off the surface;

receiving image data corresponding to one or more pictures captured by the picture-taking device of the surface in the viewing area; and

applying a perspective correction transform to the image data using the position.

16. The one or more computer-readable storage devices of claim 15, the operations further comprising:

determining a rotation of the picture-taking device relative to the surface using the position, wherein the applying of the perspective correction transform to the image data uses the rotation determined from the position.

17. The one or more computer-readable storage devices of claim 16, the operations further comprising:

receiving one or more parameters of the picture-taking device related to an angle of view of the picture-taking device, wherein the determining of the rotation of the picture-taking device is performed using the position of the reflection spot and the one or more parameters.

18. The one or more computer-readable storage devices of claim 15, wherein the applying of the perspective correction transform generates corrected image data, and the operations further comprise:

transmitting the corrected image data to the picture-taking device.

19. The one or more computer-readable storage devices of claim 15, wherein the applying of the perspective correction transform generates corrected image data, and the operations further comprise:

storing the corrected image data at a server connected by a network to the picture-taking device.

20. The one or more computer-readable storage devices of claim 15, wherein the receiving of the position of the reflection spot within the picture corresponds to receiving a tilt angle for the picture-taking device calculated from the position and the applying of the perspective correction transform to the image data uses the tilt angle.