Method and system for determining image orientation

• US 7,215,828 B2
• Filed: 02/13/2002
• Issued: 05/08/2007
• Est. Priority Date: 02/13/2002
• Status: Expired due to Fees

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Luo, Jiebo
• Primary Examiner(s): Mehta; Bhavesh M.
• Assistant Examiner(s): TUCKER, WESLEY J
• Application Number:
  US10/075,004
• Publication Number:
  US 20030152289A1
• Time in Patent Office: 1,910 Days
• Field of Search: 382289-292, 382/295, 382/296, 382/190, 382/305, 382/297
• US Class Current: 382/289
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A method for determining the orientation of a digital image, includes the steps of: employing a semantic object detection method to detect the presence and orientation of a semantic object; employing a scene layout detection method to detect the orientation of a scene layout; and employing an arbitration method to produce an estimate of the image orientation from the orientation of the detected semantic object and the detected orientation of the scene layout.

Claims

1. A method for determining the orientation of a captured digital image, comprising the steps of:

a) employing a semantic object detection method to detect the presence and orientation of a semantic object in the digital image;

b) employing a scene layout detection method to detect the orientation of a scene layout of the digital image; and

c) employing an arbitration method to produce an estimate of the image orientation by arbitrating between the orientation of the detected semantic object and the detected orientation of the scene layout.

2. The method claimed in claim 1, wherein the step of employing a semantic object detection method comprises employing a plurality of semantic object detectors to detect a plurality of semantic objects and their orientations.

3. The method claimed in claim 2, wherein the semantic objects are selected from the group of semantic objects comprising:

a human face, a human figure, clear blue sky, lawn grass, a snow field, body of open water, tree, a sign, and written text.

4. The method claimed in claim 1, wherein the scene layout detection method comprises the steps of:

a) dividing the digital image into non-overlapping image blocks;

b) computing at least one statistic for each image block;

c) forming a feature vector by concatenating the statistics computed from the image blocks; and

d) using a classifier trained with a plurality of scene prototype images to produce an estimate of the image orientation.

5. The method claimed in claim 4, wherein the statistic is related to one or more of:

color, texture, and straight lines in the digital image.

6. The method claimed in claim 1, wherein the scene layout detection method comprises the steps of:

a) extracting straight lines from the digital image;

b) computing a point of convergence (or vanishing point) from a subset of the extracted straight lines; and

c) producing an estimate of the image orientation according to the vanishing point.

7. The method claimed in claim 1, wherein the arbitration method employs a Bayes Net.

8. The method claimed in claim 1, wherein the arbitration method employs a decision tree.

9. The method claimed in claim 1, further comprising the step of:

rotating the digital image to re-orient the digital image in an upright direction.

10. A computer program product comprising computer readable storage medium having a computer program stored thereon for performing the method of claim 1.

19. The method of claim 1 wherein said captured digital image is of a natural scene.

20. The method of claim 19 wherein the orientation of the detected semantic object and the detected orientation of the scene layout are contradictory.

21. The system of claim 2, wherein:

each of said semantic object detectors detects the respective said semantic object orientation to be any one of;

upright orientation, upside-down orientation, left-to-right orientation, right-to-left orientation, and undecided orientation and

said scene layout detection method determines the orientation of the scene layout to be any one of;

upright orientation, upside-down orientation, left-to-right orientation, and right-to-left orientation.

11. A system for processing a digital color image, comprising:

a semantic object detector to determine the presence and orientation of a semantic object in the digital color image;

a scene layout detector to determine the orientation of a scene layout of the digital color image, said scene layout detector having a classifier trained with a plurality of scene prototype images;

an arbitrator responsive to the orientation of the semantic object and the orientation of the scene layout to produce an estimate of the image orientation; and

an image rotator to re-orient the digital image in the upright direction.

12. The system claimed in claim 11, wherein the step of employing a semantic object detection method comprises employing a plurality of semantic object detectors to detect a plurality of semantic objects and their orientations.

13. The system claimed in claim 12, wherein the semantic objects are selected from the group of semantic objects comprising:

a human face, a human figure, clear blue sky, lawn grass, a snow field, body of open water, tree, a sign, and written text.

14. The system claimed in claim 11, wherein the scene layout detector comprises:

a) means for dividing the digital image into non-overlapping image block;

b) means for computing at least one statistic for each image block;

c) means for forming a feature vector by concatenating the statistics computed from the image blocks; and

d) said classifier.

15. The system claimed in claim 14, wherein the statistic is related to one or more of:

color, texture, and straight lines in the digital image.

16. The system claimed in claim 11, wherein the scene layout detector, comprises:

a) means for extracting straight lines from the digital image;

b) means for computing a point of convergence (or vanishing point) from a subset of the extracted straight lines; and

c) means for producing an estimate of the image orientation according to the vanishing point.

17. The system claimed in claim 11, wherein the arbitrator is a Bayes Net.

18. The system claimed in claim 11, wherein the arbitrator includes a decision tree.

22. A system for processing a digital image comprising:

one or more semantic object detectors, each said semantic object detector being adapted to determine the presence and orientation in the digital image of a semantic object of a respective one of a plurality of different types;

a scene layout detector adapted to determine the orientation of a scene layout of the digital image; and

an arbitrator adapted to arbitrate between said determined semantic object and scene layout orientations and produce an estimate of an orientation of the digital image.

23. The system of claim 22, wherein each of said semantic object detectors determines the respective said semantic object orientation to be any one of:

upright orientation, upside-down orientation, left-to-right orientation, right-to-left orientation, and undecided orientation.

24. The method of claim 23 wherein said orientations determined by two or more of said detectors are contradictory.

25. The method of claim 22 wherein said scene layout detector has a classifier trained with a plurality of scene prototype images.

26. The method of claim 25, wherein said classifier is one of a support vector machine and a neural network.