Apparatus for detecting direction of image pickup device and moving body comprising same

• US 8,390,696 B2
• Filed: 12/18/2009
• Issued: 03/05/2013
• Est. Priority Date: 01/06/2009
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Komoto, Ayako, Kanamori, Katsuhiro, Sato, Satoshi
• Primary Examiner(s): HERNANDEZ, NELSON D
• Application Number:
  US12/935,321
• Publication Number:
  US 20110018990A1
• Time in Patent Office: 1,173 Days
• Field of Search: None
• US Class Current: 348/222.1
• CPC Class Codes: H04N 23/10 for generating image signals from different wavelengths
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

A polarization camera that can capture polarization images and a color image at the same time is used. Specifically, a clear sky polarization image, which provides information about the polarization of a part of the sky, is captured by a clear sky polarization image capturing section 100. And by reference to the information provided by a sun position determining section 1301 that determines the sun'"'"'s position at the time of shooting, a camera direction estimating section 101 determines in what direction and in what area of the whole sky the clear sky polarization image is located as a polarization pattern. Finally, information about what direction or orientation on the globe the camera (image capture device) is now facing is output. In this manner, the direction of the camera and the relative position of the camera with respect to the sun can be known without providing a sensor separately and without capturing the whole sky or the sun.

Claims

1. An image capture device direction detector for detecting the direction of the optical axis of an image capture device, the device including an image capturing section for capturing polarization images, including a polarization angle image, and a luminance image, the detector comprising:

an image processing section for generating a clear sky polarization angle image, which indicates the polarization angle of a clear sky part of the luminance image, based on the polarization images and the luminance image;

an direction estimating section for estimating the direction of the optical axis of the image capture device, which is determined by the direction of the image capturing section, by reference to the clear sky polarization angle image; and

an output section for providing information about the direction of the optical axis of the image capture device that has been estimated by the direction estimating section.

2. The image capture device direction detector of claim 1, comprising a sun position determining section for obtaining information about the sun'"'"'s position at the time of shooting,

wherein the direction estimating section estimates the direction of the optical axis of the image capture device by reference to that information.

3. The image capture device direction detector of claim 2, comprising a whole sky polarization map obtaining section to obtain a whole sky polarization map, indicating the polarization state of the sky at the time of shooting, by reference to the information about the sun'"'"'s position,

wherein the direction estimating section estimates the direction of the optical axis of the image capture device based on the clear sky polarization angle image and the whole sky polarization map.

4. The image capture device direction detector of claim 3, wherein the whole sky polarization map obtaining section retrieves a whole sky polarization map, which indicates the polarization state of the sky at the time of shooting, from a database including the whole sky polarization maps.

5. The image capture device direction detector of claim 4, comprising one or more storage devices which can retain the database referred above.

6. The image capture device direction detector of claim 2, comprising a degree of reliability determining section for determination of the degree of reliability of the result of estimation and to present the information to its user.

7. The image capture device direction detector of claim 2, wherein coordinate transformation is carried out based on the altitude and azimuth of the sun and the direction of the optical axis of the image capture device, thereby determining the sun'"'"'s position in a camera coordinate system.

8. The image capture device direction detector of claim 3, wherein the whole sky polarization map obtaining section perform calculations to generate the whole sky polarization map which indicates the polarization state of the sky at the time of shooting.

9. The image capture device direction detector of claim 4, comprising a telecommunication device to have access to the data on one or more external storage devices which retain the database referred above.

10. The image capture device direction detector of claim 6, comprising a solar altitude determining section to determine, by the solar altitude derived from the information about the sun'"'"'s position at the time of shooting, whether the result of estimation is reliable or not.

11. The image capture device direction detector of claim 1, wherein the direction estimating section calculates the direction of the clear sky part based on the polarization angle of the clear sky part, thereby estimating the direction of the image capture device.

12. The image capture device direction detector of claim 1, comprising a whole sky polarization map obtaining section to obtain a whole sky polarization map indicating the polarization state of the sky at the time of shooting,

wherein the direction estimating section operates in at least one of “

search mode” and

“

calculation mode”

, and

wherein when operating in “

search mode”

, the direction estimating section searches for the direction of the clear sky part by reference to the clear sky polarization angle image and the whole sky polarization map, and

when operating in “

calculation mode”

, the direction estimating section calculates the direction of the clear sky part based on the polarization angle of the clear sky part.

13. The image capture device direction detector of claim 1, comprising a levelness adjusting section for correcting the tilt of the image capture device.

14. The image capture device direction detector of claim 13, wherein the image capture device'"'"'s tilt includes a tilt towards the roll.

15. The image capture device direction detector of claim 14, wherein the image capture device includes a level, and

wherein the level measures the degree of levelness of the image capture device, thereby used for the compensation for the tilt of the image capture device based on the degree of levelness measured.

16. The image capture device direction detector of claim 1, comprising an angle of view measuring section for measuring the angle of view of the range of shooting, thereby determining the range of the clear sky part by the angle of view measured.

17. The image capture device direction detector of claim 1, wherein the image capturing section includes multiple polarizers with mutually different polarization principal axis angles, and

wherein the polarization images are obtained as a combination of light rays that have been transmitted through the polarizers.

18. The image capture device direction detector of claim 1, wherein the polarization images further include a degree of polarization image as well as the polarization angle image.

19. The image capture device direction detector of claim 1, wherein if the degree of polarization of the sky is equal to or higher than a reference value, the image processing section cuts out the clear sky part based on the degree of polarization, but

if the degree of polarization of the sky is lower than the reference value, the image processing section cuts out the clear sky part based on a hue and outputs the clear sky polarization angle image.

20. An image capture apparatus comprising:

an image capture device including an image capturing section for capturing polarization images, including a polarization angle image, and a luminance image; and

the image capture device direction detector of claim 1.

21. A vehicle comprising the image capture device direction detector of claim 1,

wherein the vehicle further comprises:

an image capture device including an image capturing section for capturing polarization images, including a polarization angle image, and a luminance image; and

a vehicle direction estimating section to determine the direction of the vehicle by the direction of the optical axis of the image capture device detected in accordance with a relation in direction between the vehicle and the image capture device.

22. A mobile device comprising the image capture device direction detector of claim 1,

wherein the mobile device further comprises:

an image capture device including an image capturing section for capturing polarization images, including a polarization angle image, and a luminance image; and

a mobile device direction estimating section to determine the direction of the mobile device by the direction of the optical axis of the image capture device detected in accordance with a relation in direction between the mobile device and the image capture device.

23. An image capture device comprising:

an image capturing section for capturing polarization images, including a polarization angle image, and a luminance image;

an image processing section for generating a clear sky polarization angle image, which indicates the polarization angle of a clear sky part included in the luminance image, based on the polarization images and the luminance image;

a direction estimating section for estimating the direction of the optical axis of the image capture device, which is determined by the direction of the image capturing section, by reference to the clear sky polarization angle image; and

an output section to output the image data that has been obtained by the image capturing section and providing information about the direction of the optical axis of the image capture device that has been estimated by the direction estimating section.

24. A method for detecting the direction of the optical axis of an image capture device, the method comprising the steps of:

capturing polarization images and a luminance image with the image capture device;

generating a clear sky polarization angle image, which indicates the polarization angle of a clear sky part of the luminance image, based on the polarization images and the luminance image;

estimating the direction of the image capture device by reference to the clear sky polarization angle image; and

to output the information about the direction of the optical axis of the image capture device.

25. A non-transitory computer-readable medium having stored thereon a program defined for an image capture device direction detector that detects the direction of the optical axis of an image capture device during at the time of shooting by using a celestial polarization pattern, the program being defined so as when executed by a computer causes the computer to execute the steps of:

capturing polarization images and a luminance image with the image capture device;

generating a clear sky polarization angle image, which indicates the polarization angle of a clear sky part of the luminance image, based on the polarization images and the luminance image;

estimating the direction of the optical axis of the image capture device by reference to the clear sky polarization angle image; and

to output the information about the direction of the optical axis of the image capture device.