COMPOSITE IMAGE GENERATION APPARATUS AND COMPOSITE IMAGE GENERATION PROGRAM

• US 20150258936A1
• Filed: 03/10/2015
• Published: 09/17/2015
• Est. Priority Date: 03/12/2014
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: TAKAKI, Masanari
• Granted Patent: US 9,873,379 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

In a composite image generation apparatus that is mounted in an own vehicle, each of captured images, which have been captured by a plurality of imaging units, are acquired. A disturbance level for each of the plurality of captured images that have been acquired is determined. The disturbance level indicates: whether or not a disturbance is present in the captured image; or an extent of the disturbance. In an overlapping area in which imaging areas of the plurality of captured images overlap, one or more captured images is selected from the plurality of captured images so that the area occupied by a captured image having a higher disturbance level among the plurality captured images is smaller. A composite image is generated based on the plurality of captured images. In the overlapping area, the composite image is generated by using the captured image selected based on the disturbance level.

Claims

1. A composite image generation apparatus that is mounted in an own vehicle and generates a composite image which is a combination of captured images acquired by a plurality of imaging units, the composite image generation apparatus comprising:

captured image acquiring means that acquires each of captured images that have been captured by the plurality of imaging units;

disturbance level determining means that determines a disturbance level for each of the plurality of captured images that have been acquired, the disturbance level indicating;

whether or not a disturbance is present in the captured image;

or an extent of the disturbance;

captured image selecting means that selects, in an overlapping area in which imaging areas of the plurality of captured images overlap, one or more captured images from the plurality of captured images so that the area occupied by a captured image having a higher disturbance level among the plurality captured images is smaller; and

image generating means that generates a composite image based on the plurality of captured images, and generates the composite image by using the captured image selected based on the disturbance level in the overlapping area.

2. The composite image generation apparatus according to claim 1, wherein:

the disturbance level determining means determines the type of disturbance as at least a part of the disturbance level based on a brightness value of each pixel composing each of the captured images; and

the captured image selecting means selects one or more captured images from the plurality of captured images based on the disturbance level.

3. The composite image generation apparatus according to claim 2, wherein:

the captured image selecting means selects one or more captured images that has the lowest disturbance level.

4. The composite image generation apparatus according to claim 3, wherein:

the disturbance level determining means determines the disturbance level expressed by numeric values; and

the captured image selecting means sets an occupancy ratio of each of the captured images in the overlapping area based on a ratio of the disturbance level expressed by numeric values of each of the captured images.

5. The composite image generation apparatus according to claim 4, wherein:

the image generating means generates, as the composite image, a bird'"'"'s-eye view image that is supposed to be viewed from directly above the own vehicle.

6. The composite image generation apparatus according to claim 5, further comprising:

road surface paint extracting means that extracts a road surface paint from the composite image.

7. The composite image generation apparatus according to claim 6, further comprising:

disturbance proportion calculating means that calculates, based on the disturbance level, a proportion of captured images in which a disturbance has occurred, among the plurality of captured images; and

prohibiting means that, when determined that the proportion is a threshold or higher, prohibits an operation of the road surface paint extracting means that extracts road surface paint.

8. The composite image generation apparatus according to claim 1, wherein:

the captured image selecting means selects one or more captured images that has the lowest disturbance level.

9. The composite image generation apparatus according to claim 1, wherein:

the disturbance level determining means determines the disturbance level expressed by numeric values; and

the captured image selecting means sets an occupancy ratio of each of the captured images in the overlapping area based on a ratio of the disturbance level expressed by numeric values of each of the captured images.

10. The composite image generation apparatus according to claim 1, wherein:

the image generating means generates, as the composite image, a bird'"'"'s-eye view image that is supposed to be viewed from directly above the own vehicle.

11. The composite image generation apparatus according to claim 1, further comprising:

road surface paint extracting means that extracts a road surface paint from the composite image.

12. The composite image generation apparatus according to claim 11, further comprising:

disturbance proportion calculating means that calculates, based on the disturbance level, a proportion of captured images in which a disturbance has occurred, among the plurality of captured images; and

prohibiting means that, when determined that the proportion is a threshold or higher, prohibits an operation of the road surface paint extracting means that extracts road surface paint.

13. A computer-readable storage medium storing a composite image generation program for enabling a computer to function as a composite image generation apparatus that is mounted in an own vehicle and generates a composite image which is a combination of captured images acquired by a plurality of imaging units, the composite image generation apparatus comprising:

captured image acquiring means that acquires each of captured images that have been captured by the plurality of imaging units;

disturbance level determining means that determines a disturbance level for each of the plurality of captured images that have been acquired, the disturbance level indicating;

whether or not a disturbance is present in the captured image;

or an extent of the disturbance;

captured image selecting means that selects, in an overlapping area in which imaging areas of the plurality of captured images overlap, one or more captured images from the plurality of captured images so that the area occupied by a captured image having a higher disturbance level among the plurality captured images is smaller; and

image generating means that generates a composite image based on the plurality of captured images, and generates, in the overlapping area, the composite image by using the captured image selected based on the disturbance level.

14. An image display system comprising:

a plurality of imaging units that is mounted in an own vehicle and acquires captured images; and

a composite image generation apparatus that is mounted in the own vehicle and generates a composite image which is a combination of captured images acquired by the plurality of imaging units,

a display that is mounted in the own vehicle and displays the composite image generated by the composite image generation apparatus,

the composite image generation apparatus comprising;

captured image acquiring means that acquires each of captured images that have been captured by the plurality of imaging units;

disturbance level determining means that determines a disturbance level for each of the plurality of captured images that have been acquired, the disturbance level indicating;

whether or not a disturbance is present in the captured image;

or an extent of the disturbance;

captured image selecting means that selects, in an overlapping area in which imaging areas of the plurality of captured images overlap, one or more captured images from the plurality of captured images so that the area occupied by a captured image having a higher disturbance level among the plurality captured images is smaller; and

image generating means that generates a composite image based on the plurality of captured images, and generates the composite image by using the captured image selected based on the disturbance level in the overlapping area.