Method for dynamically calibrating rotation offset in a camera system

• US 9,092,853 B2
• Filed: 10/13/2014
• Issued: 07/28/2015
• Est. Priority Date: 07/31/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Sachs, Todd S., Hunter, Kevin L., Zuliani, Marco, Molgaard, Claus
• Primary Examiner(s): HSIA, SHERRIE Y
• Application Number:
  US14/513,046
• Publication Number:
  US 20150035991A1
• Time in Patent Office: 288 Days
• Field of Search: 348/180, 348/187, 348/188, 348/583, 348/580, 702/85, 702/94, 702/95, 702150-153
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera

Abstract

A method for dynamically calibrating rotational offset in a device includes obtaining an image captured by a camera of the device. Orientation information of the device at the time of image capture may be associated with the image. Pixel data of the image may be analyzed to determine an image orientation angle for the image. A device orientation angle may be determined from the orientation information. A rotational offset, based on the image orientation angle and the device orientation angle, may be determined. The rotational offset is relative to the camera or orientation sensor. A rotational bias may be determined from statistical analysis of numerous rotational offsets from numerous respective images. In some embodiments, various thresholds and predetermined ranges may be used to exclude some rotational offsets from the statistical analysis or to discontinue processing for that image.

Claims

1. A device comprising:

a camera configured to capture images;

one or more processors; and

memory storing program instructions executable by the one or more processors to;

for a plurality of images,

obtain an image captured by the camera; and

analyze pixel data of the image to determine an image orientation for the image; and

based at least in part on a statistical analysis of at least a portion of the plurality of determined image orientations, determine an orientation bias, wherein the orientation bias is relative to the camera.

2. The device of claim 1, wherein to analyze pixel data of the image to determine the image orientation for the image, the program instructions are further executable to:

determine a horizon line for the image from application of a frequency domain technique to the captured image; and

wherein at least part of the image orientation is calculated based at least in part on an angle between the horizon line for the image and an x-axis of the image.

3. The device of claim 1, wherein the program instructions are further executable by the one or more processors to:

use the orientation bias for compensation in a horizon adjustment associated with a cropping or panorama function of the device.

4. The device of claim 3, wherein the orientation bias indicates a rotational angle of the camera with respect to the housing of the device.

5. The device of claim 1, wherein to statistically analyze at least a portion of the plurality of the determined image orientations, the program instructions are further executable by the one or more processors to:

determine whether one of the plurality of the determined image orientations exceeds a threshold; and

discard a particular one of the plurality of the determined image orientations from the statistical analysis for a particular one of the plurality of the determined image orientations that exceeds a threshold.

6. The device of claim 1, wherein the program instructions are further executable by the one or more processors to:

distinguish between images that appear to have been intentionally rotated and those that have not been intentionally rotated; and

exclude the respective image orientation from the statistical analysis for images that appear to have been intentionally rotated.

7. The device of claim 1, wherein the program instructions are further executable by the one or more processors to perform said analysis for only a limited amount of time, or as a user-selected calibration mode.

8. A method, comprising:

performing by one or more computing devices having one or more processors;

for a plurality of images;

obtaining an image captured by a camera; and

analyzing pixel data of the image to determine an image orientation for the image; and

based at least in part on a statistical analysis of at least a portion of the plurality of determined image orientations, determining an orientation bias, wherein the orientation bias is relative to the camera.

9. The method of claim 8, further comprising;

determining a horizon line for the image from application of a frequency domain technique to the captured image; and

wherein the image orientation is calculated, based at least in part on an orientation between the horizon line for the image and an x-axis of the image.

10. The method of claim 8, further comprising:

using the orientation bias for compensation in a horizon adjustment associated with a cropping or sweep panorama function of the device.

11. The method of claim 10, wherein the orientation bias indicates a rotational angle of the camera with respect to a housing of the device.

12. The method of claim 8, further comprising:

determining whether one of the plurality of image orientations exceeds a threshold; and

discarding a particular one of the plurality of image orientations from the statistical analysis for a particular one of the plurality of image orientations that exceeds a threshold.

13. The method of claim 8, further comprising:

distinguishing between images that appear to have been intentionally rotated and those that have not been intentionally rotated; and

excluding the respective image orientation from the statistical analysis for images that appear to have been intentionally rotated.

14. The method of claim 8, further comprising performing said analysis for only a limited amount of time, or as a user-selected calibration mode.

15. A non-transitory computer readable storage medium storing program instructions executable to perform:

for a plurality of images;

obtaining an image captured by the camera; and

analyzing pixel data of the image to determine an image orientation angle for the image; and

based at least in part on a statistical analysis of at least a portion of the plurality of determined image orientations, determining an orientation bias, wherein the orientation bias-is relative to the camera.

16. The non-transitory computer readable storage medium of claim 15, further comprising;

determining a horizon line for the image from application of a frequency domain technique to the captured image; and

wherein at least part of the image orientation is calculated based at least in part on an angle between the horizon line for the image and an x-axis of the image.

17. The non-transitory computer readable storage medium of claim 15, further comprising:

using the orientation bias for compensation in a horizon adjustment associated with a cropping or sweep panorama function of a device housing the camera.

18. The non-transitory computer readable storage medium of claim 15, wherein the orientation bias indicates a rotational angle of the camera with respect to a housing for the camera.

19. The non-transitory computer readable storage medium of claim 18, further comprising:

determining whether one of the plurality of the determined image orientations exceeds a threshold; and

discarding a particular one of the plurality of the determined image orientations from the statistical analysis for a particular one of the plurality of the determined image orientations that exceeds a threshold.

20. The non-transitory computer readable storage medium of claim 15, further comprising:

distinguishing between images that appear to have been intentionally rotated and those that have not been intentionally rotated; and

excluding the respective image orientation from the statistical analysis for images that appear to have been intentionally rotated.