Automatic image orientation and straightening through image analysis

• US 9,436,999 B2
• Filed: 09/18/2015
• Issued: 09/06/2016
• Est. Priority Date: 09/11/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Cieplinski, Avi, Solem, Jan Erik, Holland, Jerremy, Johnson, Garrett, Reid, Russell, Webb, Russell
• Primary Examiner(s): MOTSINGER, SEAN T
• Application Number:
  US14/859,017
• Publication Number:
  US 20160012578A1
• Time in Patent Office: 354 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 11/203 Drawing of straight lines or curves
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

Systems, methods, and computer readable media for adjusting the orientation of an image frame and a scene depicted in the image frame are described. In general, techniques are disclosed for analyzing an image with one or more feature detectors to identify features in the image. An alignment or position associated with one or more features identified in the image may be used to determine a proper orientation for the image frame. The image can then be rotated to the proper orientation. It may also be determined if a scene depicted in the image is properly aligned in the rotated image orientation. If not, alignment information associated with the identified features may be utilized to straighten the depicted scene.

Claims

1. A non-transitory program storage device, readable by a processor and comprising instructions stored thereon to cause the processor to:

receive an image;

determine an orientation for the received image based, at least in part, on one or more identified features in the received image;

identify an alignment of the one or more identified features in the received image;

determine an offset between a known typical alignment and the identified alignment for each of the one or more identified features at the determined orientation;

rotate the received image based, at least in part, on the determined offset to generate a straightened image; and

save the straightened image in a memory.

2. The non-transitory program storage device of claim 1, further comprising instructions to cause the processor to analyze the received image with one or more feature detectors in a predetermined order.

3. The non-transitory program storage device of claim 2, wherein the predetermined order comprises an order based on feature detector computational efficiency.

4. The non-transitory program storage device of claim 2, wherein the predetermined order comprises an order based on feature detector accuracy.

5. The non-transitory program storage device of claim 1, wherein the instructions to cause the processor to determine an orientation for the received image further comprise instructions to cause the processor to assign an orientation confidence value to the determined orientation.

6. The non-transitory program storage device of claim 5, wherein the orientation confidence value comprises a detected feature confidence value associated with each of the one or more identified features.

7. The non-transitory program storage device of claim 5, wherein the orientation confidence value comprises a predetermined accuracy value associated with one or more feature detectors.

8. The non-transitory program storage device of claim 1, further comprising instructions to cause the processor to rotate the received image to the determined orientation.

9. The non-transitory program storage device of claim 1, wherein the instructions to cause the processor to determine an orientation for the received image comprise instructions to cause the processor to determine a difference between a known typical position of the one or more identified features and a position of the one or more identified features in the received image.

10. The non-transitory program storage device of claim 1, wherein the instructions to cause the processor to determine an orientation for the received image comprise instructions to cause the processor to determine a difference between a known typical alignment of the one or more identified features and an alignment of the one or more identified features in the received image.

11. The non-transitory program storage device of claim 1, wherein the instructions to cause the processor to determine an orientation for the received image comprise instructions to cause the processor to utilize rotational information from an image capture device used to obtain the received image and an orientation determination for a previously analyzed image obtained by an image capture device.

12. A device comprising:

an image capture component;

a memory; and

a processor operatively coupled to the memory and the image capture component and configured to execute program code stored in the memory to;

receive an image obtained with the image capture component;

analyze the received image to identify a detected feature;

determine an orientation for the received image based, at least in part, on the detected feature;

determine an offset between a known typical alignment and an identified alignment of the detected feature at the determined orientation;

rotate the image based, at least in part, on the determined offset to generate a straightened image; and

save the straightened image in the memory.

13. The device of claim 12, wherein the detected feature comprises one or more human faces.

14. The device of claim 12, wherein the detected feature comprises intensity gradient information in the received image.

15. The device of claim 12, wherein the detected feature is identified based on a ranking of a plurality of available features.

16. The device of claim 12, wherein the program code to cause the processor to determine an orientation for the received image comprises program code to cause the processor to assign an orientation confidence value to the determined orientation.

17. The device of claim 16, further comprising program code to cause the processor to rotate the received image to the determined orientation.

18. A method, comprising:

receiving, by a processor, an image obtained with an image capture device;

analyzing, by the processor, the received image with a feature detector;

determining, by the processor, a first image orientation for the received image based, at least in part, on a detected alignment of one or more features in the received image identified by the feature detector;

rotating the received image to the first image orientation;

determining, by the processor, a second image orientation for the received image based, at least in part, on a difference between a known typical alignment and the first image orientation;

rotating, by the processor, the received image based, at least in part, on the determined second image orientation to generate a straightened image; and

saving the straightened image in a memory.

19. The method of claim 18, wherein determining a second image orientation for the received image comprises determining an in-plane rotational difference between a known typical alignment and the first image orientation.

20. The method of claim 18, wherein the feature detector comprises a feature detector configured to identify human faces.