Image capture device having tilt and/or perspective correction

• US 9,113,078 B2
• Filed: 02/07/2014
• Issued: 08/18/2015
• Est. Priority Date: 12/22/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Chen, Jason Hau-Ping, Slack, Brandon Dean, Simon, David I.
• Primary Examiner(s): NGUYEN, LUONG TRUNG
• Application Number:
  US14/175,752
• Publication Number:
  US 20140152859A1
• Time in Patent Office: 557 Days
• US Class Current: 1/1
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6815 by distinguishing pan or tilt from motion

Abstract

Methods and apparatuses are disclosed to correct for tilt and/or perspective distortion in image capture devices. In some embodiments, the method may include reading an orientation measurement associated with a relative position of an image capture device with respect to an object, determining if the orientation measurement is less than a threshold, and in the event that the orientation measurement is less than the threshold, correcting an image obtained by the image capture device. In some embodiments, the apparatus may include an image sensor, a memory coupled to the image sensor, an orientation measurement device coupled to the image sensor, and a distance measurement device coupled to the image sensor, where the image data may be stored in the memory along with a measurement from the accelerometer and along with a measurement from the distance measurement device.

Claims

1. A method of compensating for image distortion comprising:

obtaining an orientation measurement of an image capture device during an image display operation wherein a displayed image comprises data representative of a scene;

identifying one or more straight edges in the data corresponding to straight edges in the scene;

determining a difference based, at least in part, on the measured orientation of the image capture device and an orientation of at least one of the one or more straight edges in the data;

modifying the data when the difference is less than a threshold value; and

displaying the modified data.

2. The method of claim 1, wherein obtaining the orientation measurement of the image capture device comprises obtaining the orientation measurement of the image capture device along each of one or more axes relative to the image capture device.

3. The method of claim 2, wherein determining the difference comprises determining the difference based, at least in part, on the measured orientation of the image capture device along a first axis of the one or more axes and an orientation of at least one of the one or more straight edges in the data along the first axis.

4. The method of claim 2, wherein modifying the data comprises modifying the data so that the orientation of the at least one of the one or more straight edges in the data is aligned with the measured orientation of the image capture device.

5. The method of claim 2, wherein modifying the data comprises modifying the data so that the orientation of the one or more straight edges in the data is aligned near but not exactly with the measured orientation of the image capture device.

6. The method of claim 2, wherein determining the difference comprises:

determining a first angular difference between an orientation of a first of the one or more straight edges in the data and the measured orientation of the image capture device along a first of the one or more axes; and

determining a second angular difference between an orientation of the first straight edge and the measured orientation of the image capture device along a second of the one or more axes.

7. The method of claim 6, wherein modifying the data comprises:

modifying the data so that the orientation of the first straight edge is aligned with the measured orientation of the image capture device along the first axis based, at least in part, on the first angular difference; and

modifying the data so that the orientation of the first straight edge is aligned with the measured orientation of the image capture device along the second axis based, at least in part, on the second angular difference.

8. The method of claim 1, wherein modifying the data comprises displaying a distortion indication on a display, wherein the distortion indication is related to the measured orientation of the image capture device.

9. A non-transitory program storage device, readable by a processor and comprising instructions stored thereon to cause one or more processors to:

obtain an orientation measurement of an image capture device during an image display operation, wherein a displayed image comprises data representative of a scene;

identify one or more straight edges in the data corresponding to straight edges in the scene;

determine a difference based, at least in part, on the measured orientation of the image capture device and an orientation of at least one of the one or more straight edges in the data;

modify the data when the difference is less than a threshold value; and

display the modified data.

10. The non-transitory program storage device of claim 9, wherein the instructions to cause the one or more processors to obtain the orientation measurement of the image capture device comprise instructions to cause the one or more processors to obtain the orientation measurement of the image capture device along each of one or more axes relative to the image capture device.

11. The non-transitory program storage device of claim 10, wherein the instructions to cause the one or more processors to determine the difference comprise instructions to cause the one or more processors to determine the difference based, at least in part, on the measured orientation of the image capture device along a first axis of the one or more axes and an orientation of a first of the one or more straight edges in the image data along the first axis.

12. The non-transitory program storage device of claim 10, wherein the instructions to cause the one or more processors to modify the data comprise instructions to cause the one or more processors to modify the data so that the orientation of the at least one of the one or more straight edges in the data is aligned with the measured orientation of the image capture device.

13. The non-transitory program storage device of claim 10, wherein the instructions to cause the one or more processors to modify the data comprise instructions to cause the one or more processors to modify the data so that the orientation of the at least one of the one or more straight edges in the data is aligned near but not exactly with the measured orientation of the image capture device.

14. The non-transitory program storage device of claim 10, wherein the instructions to cause the one or more processors to determine the difference comprise instructions to cause the one or more processors to:

determine a first angular difference between an orientation of a first of the one or more straight edges in the data and the measured orientation of the image capture device along a first of the one or more axes; and

determine a second angular difference between an orientation of the first straight edge and the measured orientation of the image capture device along a second of the one or more axes.

15. The non-transitory program storage device of claim 14, wherein the instructions to cause the one or more processors to modify the data comprise instructions to cause the one or more processors to:

modify the data so that the orientation of the first straight edge is aligned with the measured orientation of the image capture device along the first axis based, at least in part, on the first angular difference; and

modify the data so that the orientation of the first straight edge is aligned with the measured orientation of the image capture device along the second axis based, at least in part, on the second angular difference.

16. The non-transitory program storage device of claim 9, wherein the instructions to cause the one or more processors to modify the data comprise instructions to cause the one or more processors to:

display the data and a distortion indication on a display, wherein the distortion indication is related to the measured orientation of the image capture device.

17. An image capture device, comprising:

an image sensor;

a lens assembly configured to focus light from a scene onto the image sensor;

a sensor coupled to the image capture device;

a display unit;

a memory coupled to the image sensor and the display unit; and

one or more processors coupled to the sensor, the display unit, and the memory and programmed to execute instructions stored in the memory to cause the one or more processors to—

display data representative of the scene on the display unit,

obtain a signal from the sensor representative of an orientation measurement of the image capture device during display of the data on the display unit, wherein the orientation measurement comprises an orientation measurement along each of one or more axes relative to the image capture device,

identify one or more straight edges in the data corresponding to straight edges in the scene,

determine a difference based, at least in part, on the orientation measurement and an orientation of at least one of the one or more straight edges in the data,

modify the data when the difference is less than a threshold value, and

display the modified data on the display unit.

18. The image capture device of claim 17, wherein the instructions to cause the one or more processors to determine the difference comprise instructions to cause the one or more processors to determine the difference based, at least in part, on the orientation measurement of the image capture device along a first axis of the one or more axes and an orientation of at least one of the one or more straight edges in the data along the first axis.

19. The image capture device of claim 17, wherein the instructions to cause the one or more processors to modify the data comprise instructions to cause the one or more processors to modify the data so that the orientation of a first one of the one or more straight edges in the data is aligned with the orientation measurement along a first of the one or more axes relative to the image capture device.

20. The image capture device of claim 17, wherein the instructions to cause the one or more processors to display the modified data further comprise instructions to cause the one or more processors to display a distortion indicator on the display unit, wherein the distortion indication is related to the orientation measurement of the image capture device.