Imaging apparatus, method, system integrated circuit, and program for correcting inclination of subjects in an image

• US 8,466,970 B2
• Filed: 06/05/2008
• Issued: 06/18/2013
• Est. Priority Date: 06/27/2007
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Kimura, Masayuki, Tagawa, Junichi
• Primary Examiner(s): HERNANDEZ, NELSON D
• Application Number:
  US12/664,939
• Publication Number:
  US 20100177197A1
• Time in Patent Office: 1,839 Days
• Field of Search: 348/222.1, 348/208.99, 3482081-20816, 348/239, 382/289, 382/293, 382/297, 382/276
• US Class Current: 348/208.4
• CPC Class Codes: G06T 7/77 using statistical methods
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: H04N 1/3878 Skew detection or correction
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations

Abstract

An imaging apparatus wherein a captured image is subjected to a rotational transform so as to correct the inclination of one or more subjects in the captured image. The imaging apparatus defines, with reference to a physical inclination detected by a sensor, an angle range in which the image is to be processed. The imaging apparatus then processes the captured image and detects lines therein and inclination angles thereof. The imaging apparatus then creates histograms indicative of frequency distributions of the inclination angles of those lines, and selects, in the created histograms, an inclination angle having a frequency that satisfies a predetermined criterion, as a rotational angle to be used for the rotational transform.

Claims

1. An imaging apparatus for correcting inclination of one or more subjects in a captured image by applying a rotational transform to the captured image, the imaging apparatus comprising:

an imaging unit operable to capture an image;

a sensor operable to detect a physical inclination angle of the imaging unit;

an image processing unit operable to determine an angle range that includes the physical inclination angle detected by the sensor when the imaging unit has captured the image and a width determined from measurement errors of the sensor, and then to detect a plurality of angular components in one or more subjects appearing in the image captured by the imaging unit, the plurality of angular components individually falling within the angle range; and

a correction processing unit operable to measure a frequency distribution of the plurality of angular components to select one of angular components as a rotation angle for the rotational transform based on the frequency distribution, and then to apply the rotational transform to the image.

2. The imaging apparatus of claim 1, wherein

the plurality of angular components are inclination angles of lines appearing in the image,

the correction processing unit creates a histogram indicating the frequency distribution, and

the rotation angle selected by the correction processing unit has a specific frequency in the histogram created by the correction processing unit, the specific frequency satisfying a criterion for determining an angular component probably equal to a true inclination angle by which the imaging unit was tilted when having captured the image.

3. The imaging apparatus in claim 2, wherein

the criterion indicates that the specific frequency is a highest frequency in the histogram.

4. The imaging apparatus in claim 2, wherein

the imaging apparatus further includes a position detection unit operable to detect a position of the imaging apparatus, wherein

when none of the plurality of angular components has the specific frequency, and the position detected by the position detection unit is a predetermined position, the correction processing unit selects one of the plurality of angular components that has the highest frequency in the histogram, as the rotation angle for rotational transform.

5. The imaging apparatus in claim 2 further comprising:

a storage unit storing data for error distributions, each showing uncertainty of a different inclination angle to be detected by the sensor; and

an error distribution obtaining unit operable to refer to the data from the storage unit to obtain one of the error distributions suitable for an inclination angle actually detected by the sensor, wherein

the criterion indicates that a greatest inclination likelihood is calculated from the specific frequent and the one of the error distributions obtained by the error distribution obtaining unit.

6. The imaging apparatus in claim 5, wherein

the error distributions contained in the data stored in the storage unit include both first and second distributions for the imaging apparatus being at rest and in motion respectively, and

the error distribution obtaining unit (i) determines a motion status of the imaging apparatus from the physical inclination angle detected by the sensor, and (ii) obtains the first and second distributions from the storage unit if a motion status shows that the imaging apparatus is at rest and in motion, respectively.

7. The imaging apparatus in claim 2 further comprising:

a storage unit storing data on an arithmetic expression for a relationship between error distributions of the sensor and inclination angles to be detected by the sensor; and

an error distribution calculation unit operable to refer to the data from the storage unit to use the arithmetic expression to calculate one of the error distributions suitable for an inclination angle detected by the sensor, wherein

the criterion indicates that a greatest inclination likelihood is calculated from the specific frequency and the one of the error distributions calculated by the error distribution calculation unit.

8. The imaging apparatus in claim 7, wherein

the data stored in the storage unit contains both first and second expressions for the relationships of the imaging apparatus being at rest and in motion, respectively, and

the error distribution calculation unit (i) determines a motion status of the imaging apparatus from the physical inclination angle detected by the sensor, and (ii) uses the first and second expressions to calculate one of the error distributions if the motion status shows that the imaging apparatus is at rest and in motion, respectively.

9. The imaging apparatus in claim 2, wherein

the correction processing unit, when creating the histogram, assigns weights to frequencies of the plurality of angular components, the weights increasing with lengths of the lines corresponding to the plurality of angular components.

10. The imaging apparatus in claim 2, wherein

the correction processing unit, when creating the histogram, assigns weights to frequencies of the plurality of angular components, the weights increasing with a decrease in inclination from the vertical direction of the lines corresponding to the plurality of angular components.

11. The imaging apparatus in claim 2, wherein

the correction processing unit, when creating the histogram, assigns weights to frequencies of the plurality of angular components, the weights increasing with a decrease in differences between the physical inclination angle detected by the sensor and the inclination angles of the lines corresponding to the plurality of angular components.

12. The imaging apparatus in claim 1, wherein when the frequency distribution shows that none of the plurality of angular components is to be selected, the correction processing unit selects the physical inclination angle detected by the sensor as the rotation angle for the rotational transform.

13. The imaging apparatus in claim 1, wherein

the image processing unit further includes a detection unit operable to detect a particular subject in image, and

when the frequency distribution shows that none of the plurality of angular components is to be selected, the correction processing unit selects an inclination angle of the particular subject as the rotation angle for the rotational transform.

14. The imaging apparatus in claim 1, further comprising

a distortion correction unit operable to correct a distortion of the image before the process by the image processing unit to detect the plurality of angular components in the image.

15. A method for correcting inclination of one or more subjects in a captured image by applying a rotational transform to the captured image, comprising:

detecting a physical inclination angle of an imaging unit using a sensor when the image unit captures an image;

determining an angle range that includes the physical inclination angle and a width determined from measurement errors of the sensor;

detecting a plurality of angular components in one or more subjects appearing in the image, the plurality of angular components individually falling within the angle range;

measuring a frequency distribution of the plurality of angular components;

selecting one of the plurality of angular components as a rotation angle for rotational transform based on the frequency distribution; and

applying the rotational transform to the image.

16. A system integrated circuit, housed in an imaging apparatus, operable to correct inclination of one or more subjects in a captured image by applying a rotational transform to the captured image, the system integrated circuit comprising:

an imaging unit operable to capture an image;

a sensor operable to detect a physical inclination angle of the imaging unit;

an image processing unit operable to determine an angle range that includes the physical inclination angle detected by the sensor when the imaging unit has captured the image and a width determined from measurement errors of the sensor, and then to detect a plurality of angular components in one or more subjects appearing in the image captured by the imaging unit, the plurality of angular components individual falling within the angle range; and

a correction processing unit operable to measure a frequency distribution of the plurality of angular components, to select one of the plurality of angular components as a rotation angle for the rotational transform based on the frequency distribution, and then apply the rotational transform to the image.

17. A non-transitory computer-readable recording medium storing a program for correction processing of inclination of one or more subjects in a captured image by applying a rotational transform to the captured image, the computer program causing a computer to perform the steps of:

detecting a physical inclination angle of an imaging unit by a sensor when the imaging unit captures an image;

determining an angle range that includes the physical inclination angle and a width determined from measurement errors of the sensor;

detecting a plurality of angular components in one or more subjects appearing in the image, the plurality of angular components individually falling within the angle range;

measuring a frequency distribution of the plurality of angular components;

selecting one of the plurality of angular components as a rotation angle for the rotational transform based on the frequency distribution; and

applying the rotational transform to the image.

18. An imaging apparatus for correcting inclination of one or more subjects in a captured image by applying a rotational transform to the captured image, the imaging apparatus comprising:

an imaging unit operable to capture an image;

a sensor operable to detect a physical inclination angle of the imaging unit;

an image processing unit operable to detect a plurality of angular components in one or more subjects appearing in the image captured by the imaging unit; and

a correction processing unit operable to determine an angle range that includes the physical inclination angle detected by the sensor when the imaging unit has captured the image and a width determined from measurement errors of the sensor, to measure a frequency distribution of a portion of the plurality of angular components, the portion including angular components individually falling within the angle range, to select one of the plurality of angular components as a rotation angle for the rotational transform based on the frequency distribution, and then to apply the rotational transform to the image.

19. The imaging apparatus of claim 18, wherein

the plurality of angular components are angles of luminance gradients appearing in the captured image,

the correction processing unit creates a histogram indicating the frequency distribution, and

the rotation angle selected by the correction processing unit has a specific frequency in the histogram created by the correction processing unit, the specific frequency satisfying a criterion for determining an angular component probably equal to a true inclination angle by which the imaging unit was tilted when having captured the image.