IMAGE PROCESSOR, DRIVING ASSISTANCE SYSTEM, AND OUT-OF-POSITION DETECTING METHOD

• US 20080170122A1
• Filed: 01/10/2008
• Published: 07/17/2008
• Est. Priority Date: 01/11/2007
• Status: Active Grant

• Current Assignee: Sanyo Electric Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Sanyo Electric Company Limited (Panasonic Holdings Corporation)
• Inventors: Okuda, Kozo, HONGO, Hitoshi
• Granted Patent: US 8,184,160 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: B60W 2050/0063 Manual parameter input, manual setting means, manual initialising or calibrating means
• CPC Class Codes: B60W 2050/0075 Automatic parameter input, automatic initialising or calibrating means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

A driving assistance system that projects images imaged by multiple vehicle cameras onto a plane, and synthesizes projection images from images obtained at different times to accurately determine whether the vehicle cameras have moved out of their proper positions or imaging directions. The system includes: first and second in-vehicle cameras that capture first and second images, respectively. The cameras capture overlapping regions respectively in different directions. An image converter converts the first and second images into first and second projection images projected onto a plane. A solid object detection unit determines whether a solid object exists in the overlapping region. When no solid object exists, the images of the overlapping regions within the first and second projection images are compared to generate a comparison value. The comparison value is in turn compared with a threshold value for the determination of vehicle camera movement out of its proper position.

Claims

1. An image processor, comprising:

an image converter configured to convert a first image into a first projection image projected on a plane and to convert a second image into a second projection image projected on a plane, the first image acquired by imaging a first target region around a vehicle captured with a first vehicle camera, the second image acquired by imaging a second target region captured with a second vehicle camera, wherein the second target region overlaps the first target region;

an image synthesizer configured to synthesize an image from the first projection image and the second projection image;

a solid object determination unit configured to detect a solid object in the overlapping region from the first projection image and the second projection image;

a difference calculator configured to calculate, when the solid object determination unit detects the absence of a solid object, a difference between an image corresponding to the overlapping region within the first projection image, and an image corresponding to the overlapping region within the second projection image; and

a detector configured to detect movement of at least the first vehicle camera or the second vehicle camera out of position by comparing between the calculated difference and a threshold value.

2. The image processor as claimed in claim 1,

wherein the image synthesizer acquires first images from the first vehicle camera at a first and a second timing;

the solid object determination unit comprises;

a motion vector acquisition unit configured to acquire motion vectors based on an overlapping region image within the first projection image converted from the first image acquired at the first timing and the first projection image converted from the first image acquired at the second timing; and

a motion vector determination unit configured to compare the motion vectors acquired by the motion vector acquisition unit, and to detect a solid object in the overlapping region.

3. The image processor as claimed in claim 2, wherein the motion vector acquisition unit detects discriminative points common to the image corresponding to the overlapping region within the first projection image acquired at the first timing, and to the first projection image converted from the first image acquired at the second timing, and acquires motion vectors based on the discriminative points.

4. The image processor as claimed in claim 3, wherein a discriminative point corresponds to a pixel at an edge region defined by having a large difference from surrounding pixels for each image corresponding to the overlapping region within the first projection image acquired at the first timing, and the first projection image converted from the first image acquired at the second timing.

5. The image processor as claimed in claim 2, wherein the motion vector determination unit receives the motion vectors, and determines whether differences in direction and length among the motion vectors are within tolerance values.

6. The image processor as claimed in claim 1,

wherein the image synthesizer acquires the first images from the first vehicle camera at a first and a second timing;

the image processor further comprises a moving distance obtaining unit configured to obtain a moving distance of a vehicle between the first timing and the second timing; and

wherein the solid object determination unit further comprises;

an identifier configured to identify a comparison region within the first projection image converted from the first image acquired at the second timing, based on the moving distance, wherein the comparison region corresponds to the overlapping region within the first projection image converted from the first image acquired at the first timing; and

a determination unit configured to detect a solid object in the overlapping region, based on a difference between an image corresponding to the comparison region identified by the identifier, and an image corresponding to the overlapping region.

7. An image processor, comprising:

an image converter configured to convert a first image into a first projection image projected on a plane, the first image acquired by imaging a first target region around a vehicle captured with a first vehicle camera, and to convert a second image into a second projection image projected on a plane, the second image acquired by imaging a second target region having a region that overlaps the first target region, the second image captured with a second vehicle camera;

an image synthesizer configured to generate a synthesis image from the first and second projection images;

a difference calculator configured to calculate a difference between images that correspond to the overlapping regions within the first and second projection images; and

a detector configured to detect movement of at least the first vehicle camera or the second vehicle camera out of position, based on the number of times that the difference calculated by the difference calculator consecutively is equal or greater to a threshold value.

8. A driving assistance system, comprising:

a first vehicle camera configured to image a first target region around a vehicle;

a first acquisition unit configured to acquire a first image imaged by the first vehicle camera;

a second vehicle camera configured to image a second target region around the vehicle; and

a second acquisition unit configured to acquire a second image imaged by the second vehicle camera;

an image converter configured to convert a first image into a first projection image projected on a plane and to convert a second image into a second projection image projected on a plane, the first image acquired by imaging a first target region around a vehicle captured with a first vehicle camera, the second image acquired by imaging a second target region captured with a second vehicle camera, wherein the second target region overlaps the first target region;

an image synthesizer configured to synthesize an image from the first projection image and the second projection image;

a solid object determination unit configured to detect a solid object in the overlapping region from the first projection image and the second projection image;

a difference calculator configured to calculate, when the solid object determination unit determines the absence of a solid object, a difference between an image corresponding to the overlapping region within the first projection image, and an image corresponding to the overlapping region within the second projection image; and

a detector configured to detect movement at least the first vehicle camera or the second vehicle camera out of position by comparing between the calculated difference and a threshold value.

9. An out-of-position detecting method, comprising:

converting a first image into a first projection image projected on a plane, the first image acquired by imaging a first target region around a vehicle from a first vehicle camera;

converting a second image into a second projection image projected on a plane, the second image acquired by imaging a second target region having a region that overlaps the first target region, from a second vehicle camera;

generating a synthesis image from the first and second projection images;

detecting a solid object in the overlapping region;

calculating, when no solid object is detected, a difference between an image corresponding to the overlapping region within the first projection image, and an image corresponding to the overlapping region within the second projection image; and

detecting movement of at least the first vehicle camera or the second vehicle camera out of position by comparing the calculated difference and a threshold value.

10. The method as claimed in claim 9, wherein the image synthesizer acquires the first images from the first vehicle camera at a first and a second timing;

and

wherein the method of detection of a solid object in the overlapping region comprises;

acquiring motion vectors based on an image corresponding to the overlapping region within the first projection image converted from the first image acquired at a first timing and the first projection image converted from the first image acquired at the second timing; and

comparing the motion vectors to detect a solid object in the overlapping region.

11. The method as claimed in claim 10, wherein the acquiring motion vectors step comprises detecting discriminative points common to the image corresponding to the overlapping region within the first projection image acquired at the first timing, and the first projection image converted from the first image acquired at the second timing, and acquires motion vectors from the discriminative points.

12. The method as claimed in claim 11, wherein a discriminative point corresponds to a pixel at an edge region defined by having a large difference from surrounding pixels for each image corresponding to the overlapping region within the first projection image acquired at the first timing, and the first projection image converted from a first image acquired at the second timing.

13. The method as claimed in claim 10, wherein

the comparison of motion vector comprises receiving motion vectors, and comparing differences in direction and length between the motion vectors with tolerance values.

14. The method as claimed in claim 9, wherein

the image synthesis includes acquiring the first images from the first vehicle camera at a first and a second timing;

the method further comprises obtaining a moving distance of a vehicle between the first timing and the second timing; and

the detection of a solid object comprises;

identifying a comparison region within the first projection image converted from the first image acquired at the second timing, from the moved distance, the comparison region corresponding to the overlapping region within the first projection image converted from the first image acquired at the first timing; and

detecting a solid object in the overlapping region, from comparing an image corresponding to the comparison region identified by the identifier with an image corresponding to the overlapping region.

15. An out-of-position detecting method, comprising:

obtaining a first image of a first target region around a vehicle with a first vehicle camera, and converting the first image into a first projection image projected onto a plane;

obtaining a second image of a second overlapping target region around the vehicle with a second vehicle camera, and converting the second image into a second projection image projected on a plane,

synthesizing the first projection image and the second projection image into a synthesis image;

comparing the overlapping regions of the first and second projection images to generate comparison results; and

detecting movement of at least the first vehicle camera or the second vehicle camera out of position based on the number of times that the calculated difference is consecutively equal or greater to a threshold value.