Vehicle periphery monitoring device and vehicle periphery image display method

• US 9,247,217 B2
• Filed: 07/27/2010
• Issued: 01/26/2016
• Est. Priority Date: 07/29/2009
• Status: Active Grant

• Current Assignee: Clarion Co., Ltd. (Faurecia SA)
• Original Assignee: Clarion Co., Ltd. (Faurecia SA)
• Inventors: Okuyama, Yoshitaka
• Primary Examiner(s): Ustaris, Joseph
• Assistant Examiner(s): VOLENTINE, REBECCA A
• Application Number:
  US13/259,134
• Publication Number:
  US 20120026333A1
• Time in Patent Office: 2,009 Days
• US Class Current: 1/1
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 2300/103 using camera systems provided with artificial illumination device, e.g. IR light source
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/106 using night vision cameras
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/8053 for bad weather conditions or night vision
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

From a plurality of imaging units (110) which capture images of a vehicle periphery and from an image taken by any one of the plurality of imaging units (any one of 111, 121, 131, 141), a value corresponding to brightness of the image is calculated. Based on the calculated value, brightness of images obtained by the plurality of imaging units (110) is corrected by a brightness correction unit (120). After a predetermined coordinate transformation is performed on the brightness corrected images, the images are combined into a single image, and the single image is displayed together with the image taken by the one imaging unit (any one of 111, 121, 131, 141).

Claims

1. A vehicle periphery monitoring device comprising:

a plurality of imaging units mounted on a vehicle and configured to capture images of a vehicle periphery;

an image brightness calculation processor configured to calculate a value corresponding to brightness of images taken by the plurality of imaging units;

a brightness correction unit, said brightness correction unit being configured to correct the brightness of the images obtained by the plurality of imaging units on the basis of the value calculated by the image brightness calculation processor;

an image combining processor configured to combine the plurality of images obtained by the plurality of imaging units and corrected by the brightness correction unit into a single image;

an image convertor that converts the single image formed by the image combining processor into an overhead image viewed from right above the vehicle;

a traveling direction judgment processor configured to judge a traveling direction of a vehicle;

an imaging-unit-selecting processor configured to select one image from the images taken by the plurality of imaging units on the basis of a judgment result of the traveling direction judgment processor; and

an image display unit that simultaneously displays the overhead image formed by the image convertor and one image selected by the imaging-unit-selecting processor;

wherein

the image brightness calculation processor calculates a value corresponding to brightness of the image imaged by the imaging unit corresponding to a traveling direction of the vehicle based on the determination result of the traveling direction judgment processor and selected by the imaging-unit-selecting processor,

the brightness correction unit corrects brightness of the images taken by the plurality of imaging units other than the imaging unit corresponding to the traveling direction based on the difference between the value corresponding to the brightness of the image taken by the imaging unit in the traveling direction and calculated by the image brightness calculation processor and the values corresponding to brightness of the others of the plurality of the imaging units for an image area which is overlapped on the image area of the traveling direction, and

the image display unit simultaneously displays the overhead image combined by the image combining processor and converted by the image convertor and the image selected by the imaging-unit-selecting processor, the brightness of the combined and converted overhead image being corrected.

2. The vehicle periphery monitoring device according to claim 1,

wherein

the imaging-unit-selecting processor selects a first imaging unit from the plurality of imaging units which capture images in three or more directions and which are mounted so that ranges captured by at least two of the plurality of imaging units partly overlap each other,

the image brightness calculation processor calculates a value corresponding to brightness of a first overlapping imaging range included in an image taken by the selected first imaging unit and in an image taken by a second imaging unit which shares the first overlapping imaging range with the selected first imaging unit, or calculates a value corresponding to brightness of a neighborhood of the first overlapping imaging range,

the brightness correction unit corrects brightness of the image taken by the second imaging unit on the basis of the calculated value,

based on the brightness corrected result, the image brightness calculation processor further calculates a new value corresponding to brightness of a second overlapping imaging range or corresponding to brightness of a neighborhood of the second overlapping imaging range, from

the brightness corrected image and an image taken by a third imaging unit and sharing the second overlapping imaging range with the brightness corrected image,

the brightness correction unit further corrects brightness of the image taken by the third imaging unit on the basis of the calculated new value, and

the image brightness calculation processor and the brightness correction unit sequentially repeat the calculation of brightness and the correction of brightness in the same manner thereafter.

3. The vehicle periphery monitoring device according to claim 1,

wherein

the image brightness calculation processor calculates a value corresponding to brightness of a first overlapping imaging range included in an image taken by a first imaging unit of the plurality of imaging units, which capture images in three or more directions and which are mounted so that ranges captured by at least two of the plurality of imaging units partly overlap each other, and included in an image taken by a second imaging unit sharing the first overlapping imaging range with the first imaging unit or calculates a value corresponding to brightness of a neighborhood of the first overlapping imaging range,

the brightness correction unit corrects brightness of the image taken by the second imaging unit on the basis of the calculated value,

based on the brightness corrected result, the image brightness calculation processor further calculates a new value corresponding to brightness of a second overlapping imaging range or corresponding to brightness of a neighborhood of the second overlapping imaging unit, from the brightness corrected image and an image taken by a third imaging unit and sharing the second overlapping imaging range with the brightness corrected image,

the brightness correction unit further corrects brightness of the image taken by the third imaging unit on the basis of the calculated new value, and

the image brightness calculation processor and the brightness correction unit sequentially repeat the calculation of brightness and the correction of the brightness in the same manner thereafter.

4. A vehicle periphery image display method for displaying, in one display screen, an image obtained by combining a plurality of images of a vehicle periphery taken by a plurality of imaging units and an image taken by one of the plurality of imaging units, comprising the steps of:

judging a traveling direction of a vehicle;

selecting one image from a plurality of images captured for a vehicle periphery on the basis of the judgment result of the traveling direction;

calculating a value corresponding to brightness for each of the plurality of images;

correcting brightness of the plurality of images, except the one image selected based on the traveling direction, on the basis of difference between the calculated value of the one image and each value of the plurality of images which has an overlapping portion with the one image;

performing a predetermined coordinate transformation process on the plurality of images, the brightness of which is corrected;

combining the plurality of coordinate-transformed images into a single image;

converting the single image into an overhead image viewed from right above the vehicle;

displaying the overhead image and the one image simultaneously.

5. The vehicle periphery image display method according to claim 4, further comprising the steps of:

selecting a first image from a plurality of images captured for three or more directions and set so that imaging ranges of at least two of the plurality of images partly overlap each other;

calculating a value corresponding to brightness of a first overlapping imaging range included in the selected first image and in a second image sharing the first overlapping imaging area with the selected first image, or a value corresponding to brightness of a neighborhood of the first overlapping imaging range;

correcting brightness of the second image on the basis of the calculated value;

calculating, based on the brightness corrected result, a new value corresponding to brightness of a second overlapping imaging range or corresponding to brightness of a neighborhood of the second overlapping imaging area from the brightness corrected image and a third image sharing the second overlapping imaging range with the brightness corrected image;

correcting brightness of the third image on the basis of the calculated new value; and

sequentially repeating the calculation of brightness and the correction of brightness in the same manner thereafter.

6. The vehicle periphery image display method according to claim 4, further comprising the steps of:

calculating a value corresponding to brightness of a first overlapping imaging range included in a first image of a plurality of images captured for three or more directions and set so that imaging ranges of at least two of the plurality of images partly overlap, and included in a second image in which the first overlapping imaging area that overlaps the first image is captured, or a value corresponding to brightness of a neighborhood of the first overlapping imaging range;

correcting brightness of the different image on the basis of the calculated value;

calculating, based on the brightness corrected result, a new value corresponding to brightness of a second overlapping imaging area or corresponding to brightness of a neighborhood of the second overlapping imaging range, from the brightness corrected image and a third image sharing the second overlapping imaging range with the brightness corrected image;

correcting brightness of the third image on the basis of the calculated new value; and

sequentially repeating the calculation of brightness and the correction of brightness in the same manner thereafter.