Image-Processing System and Image-Processing Method

• US 20090097708A1
• Filed: 10/02/2008
• Published: 04/16/2009
• Est. Priority Date: 10/15/2007
• Status: Active Grant

• Current Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.)
• Original Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.)
• Inventors: Mizuta, Masaki
• Granted Patent: US 8,155,385 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2300/60 characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images

Abstract

A vehicle-periphery-image-providing system may include an image-capturing unit, a viewpoint-change unit, an image-composition unit, an object-decttion unit, a line-width-setting unit, and a line-selection unit. The image-capturing units, such as cameras, capture images outside a vehicle periphery and generate image-data items. The viewpoint-change unit generates a bird'"'"'s-eye-view image for each image-data item based on the image-data item so that end portions of the real spaces corresponding to two adjacent bird'"'"'s-eye-view images overlap each other. The image-composition unit generates a bird'"'"'s-eye-view-composite image by combining the bird'"'"'s-eye-view images according to a predetermined layout. The object-detection unit detects an object existing in the real space corresponding to a portion where the bird'"'"'s-eye-view images of the bird'"'"'s-eye-composite image are joined to each other. The line-width-setting unit sets the width of the line image corresponding to the joining portion. The line-selection unit adds a line image having the set width to an overlap portion of one of the bird'"'"'s-eye-view images.

Claims

1. An image-processing system operative to process image data obtained by capturing images outside a periphery of a vehicle, the image-processing system comprising:

a plurality of image-capturing units that is affixed to the vehicle and that generates image-data items by capturing images outside the periphery of the vehicle;

a bird'"'"'s-eye-view-image-drawing unit configured to generate a bird'"'"'s-eye-view image by determining a viewpoint above the vehicle for each of the image-data items generated by the image-capturing units based on the image-data item so that end portions of real spaces corresponding to two adjacent bird'"'"'s-eye-view images overlap each other;

an image-composition unit configured to generate a bird'"'"'s-eye-view-composite image by combining the bird'"'"'s-eye-view images with one another according to a predetermined layout;

an object-detection unit configured to detect at least one object existing in at least one real space corresponding to at least one joining portion where the bird'"'"'s-eye-view images included in the bird'"'"'s-eye-view-composite image are joined to each other;

a border-width-setting unit configured to determine a border image based on the overlap portions of the bird'"'"'s-eye-view images included in the bird'"'"'s-eye-view-composite image, the overlap portions being included in the joining portion, and to change a width of the border image when the object-detection unit detects at least one object existing in a real space corresponding to the overlap portion; and

an image-addition unit configured to generate a border-and-bird'"'"'s-eye-view-composite image in the joining portion by superimposing a border image having a width set by the border-width-setting unit on the overlap portion of one of the bird'"'"'s-eye-view images.

2. The image-processing of claim 1, wherein the object-detection unit includes a sensor that is provided for each of the at least one real space corresponding to the at least one joining portion, the sensor operative to detect the at least one object existing in the real space, and

wherein the border-width-setting unit sets the width of the border image to a value smaller than an initial value when the object-detection unit detects the object existing in the real space corresponding to the joining portion.

3. The image-processing system of claim 1, wherein, when generating the overlap portions of the bird'"'"'s-eye-view images, the image-composition unit determines a bird'"'"'s-eye-view image corresponding to the image-data item generated by the image-capturing unit having a narrow view angle of the image-capturing units generating the image-data items corresponding to the bird'"'"'s-eye-view images to be the other bird'"'"'s-eye-view image, and draws the overlap portion of the other bird'"'"'s-eye-view image with priority.

4. The image-processing system of claim 1, further comprising:

a frame memory configured to store image data corresponding to an image for display,

wherein the image-composition unit stores image data corresponding to each of the bird'"'"'s-eye-view images drawn by the image-drawing unit in a predetermined storage area determined for each of the bird'"'"'s-eye-view images, the predetermined storage area being provided in the frame memory, and

wherein the image-addition unit stores image data corresponding to the border image in a predetermined storage area determined for each of the border images, the predetermined storage area being provided in the frame memory.

5. The image-processing system of claim 4, further comprising:

a mapping memory configured to store data on a mapping table showing an association between a position of each of pixels of the bird'"'"'s-eye-view image generated by the image-drawing unit and a storage area provided in the frame memory,

wherein the image-composition unit stores image data on each of pixels corresponding to the bird'"'"'s-eye-view image drawn by the image-drawing unit in a predetermined storage area provided in the frame memory based on the mapping table.

6. The image-processing system of claim 1, wherein the plurality of image-capturing units comprises:

a first image-capturing unit capturing an image of an area ahead of the vehicle;

a second image-capturing unit capturing an image of an area on the right of the vehicle;

a third image-capturing unit capturing an image of an area in the rear of the vehicle; and

a fourth image-capturing unit capturing an image of an area on the left of the vehicle.

7. An image-processing system processing image data obtained by capturing images outside a periphery of a vehicle, the image-processing system comprising:

a plurality of image-capturing units that is affixed to the vehicle and that generates image-data items by capturing images outside the periphery of the vehicle;

a bird'"'"'s-eye-view-image-drawing unit configured to generate a bird'"'"'s-eye-view image by determining a viewpoint above the vehicle for each of the image-data items generated by the image-capturing units based on the image-data item so that end portions of real spaces corresponding to two adjacent bird'"'"'s-eye-view images overlap each other;

an image-composition unit configured to generate a bird'"'"'s-eye-view-composite image by combining the bird'"'"'s-eye-view images with one another according to a predetermined layout;

a posture-change-detection unit configured to detect an amount of change in a posture of the vehicle;

a border-width-setting unit configured to determine a border image based on the overlap portions included in a portion where the bird'"'"'s-eye-view images included in the bird'"'"'s-eye-view-composite image are joined to each other and to change a width of the border image based on the detected vehicle-posture-change amount; and

an image-addition unit configured to draw a border-and-bird'"'"'s-eye-view-composite image in the joining portion by superimposing a border image having a width set by the border-width-setting unit on the overlap portion of one of the bird'"'"'s-eye-view images.

8. The image-processing system of claim 7, wherein, when drawing the overlap portions of the bird'"'"'s-eye-view images, the image-composition unit determines a bird'"'"'s-eye-view image corresponding to the image-data item generated by the image-capturing unit having a narrow view angle of the image-capturing units generating the image-data items corresponding to the bird'"'"'s-eye-view images to be the other bird'"'"'s-eye-view image, and generates the overlap portion of the other bird'"'"'s-eye-view image with priority.

9. The image-processing system of claim 7, further comprising:

a frame memory configured to store image data corresponding to an image for display,

wherein the image-composition unit stores image data corresponding to each of the bird'"'"'s-eye-view images generated by the image-drawing unit in a predetermined storage area determined for each of the bird'"'"'s-eye-view images, the predetermined storage area being provided in the frame memory, and

wherein the image-addition unit stores image data corresponding to the border image in a predetermined storage area determined for each of the border images, the predetermined storage area being provided in the frame memory.

10. The image-processing system of claim 9, further comprising:

a mapping memory configured to store data on a mapping table showing an association between a position of each of pixels of the bird'"'"'s-eye-view image generated by the image-drawing unit and a storage area provided in the frame memory,

wherein the image-composition unit stores image data corresponding to each of the pixels corresponding to the bird'"'"'s-eye-view image generated by the image-drawing unit in a predetermined storage area provided in the frame memory based on the mapping table.

11. The image-processing system of claim 7, wherein the plurality of image-capturing units includes:

a first image-capturing unit capturing an image of an area ahead of the vehicle;

a second image-capturing unit capturing an image of an area on the right of the vehicle;

a third image-capturing unit capturing an image of an area in the rear of the vehicle; and

a fourth image-capturing unit capturing an image of an area on the left of the vehicle.

12. An image-processing method to process image data obtained by capturing images outside a periphery of a vehicle, the image-processing method comprising the steps of:

capturing images outside the periphery of the vehicle by using a plurality of image-capturing units affixed to the vehicle and generating image-data items;

generating a bird'"'"'s-eye-view image by determining a viewpoint above the vehicle for each of the image-data items generated at the image-capturing step based on the image-data item so that end portions of real spaces corresponding to two adjacent bird'"'"'s-eye-view images overlap each other;

generating a bird'"'"'s-eye-view-composite image by combining the bird'"'"'s-eye-view images with one another according to a predetermined layout;

detecting at least one object existing in at least one real space corresponding to at least one portion where the bird'"'"'s-eye-view images included in the bird'"'"'s-eye-view-composite image are joined to each other;

generating a border image based on the overlap portions of the bird'"'"'s-eye-view images included in the bird'"'"'s-eye-view-composite image, the overlap portions being included in the joining portion, and changing a width of the border image when at least one object existing in a real space corresponding to the overlap portion is detected at the object-detection step; and

generating a border-and-bird'"'"'s-eye-view-composite image in the joining portion by superimposing a border image having a width set at the border-width-setting step on the overlap portion of one of the bird'"'"'s-eye-view images.

13. An image-processing method provided to process image data obtained by capturing images outside a periphery of a vehicle, the image-processing method comprising the steps of:

capturing images outside the periphery of the vehicle using a plurality of image-capturing units affixed to the vehicle and generating image-data items;

generating a bird'"'"'s-eye-view image by determining a viewpoint above the vehicle for each of the image-data items generated at the image-capturing step based on the image-data item so that end portions of real spaces corresponding to two adjacent bird'"'"'s-eye-view images overlap each other;

generating a bird'"'"'s-eye-view-composite image by combining the bird'"'"'s-eye-view images with one another according to a predetermined layout;

detecting an amount of change in a posture of the vehicle;

generating a border image based on the overlap portions included in a portion where the bird'"'"'s-eye-view images included in the bird'"'"'s-eye-view-composite image are joined to each other and changing a width of the border image based on the detected vehicle-posture-change amount; and

generating a border-and-bird'"'"'s-eye-view-composite image in the joining portion by superimposing a border image having a width set at the border-width-setting step on the overlap portion of one of the bird'"'"'s-eye-view images.