IMAGE GENERATION DEVICE

• US 20140063197A1
• Filed: 05/23/2012
• Published: 03/06/2014
• Est. Priority Date: 06/09/2011
• Status: Active Grant

• Current Assignee: Aisin Seiki Co., Ltd. (Aisin Corp.)
• Original Assignee: Aisin Seiki Co., Ltd. (Aisin Corp.)
• Inventors: Yamamoto, Kinji, Watanabe, Kazuya, Miura, Kenji, Kadowaki, Jun, Takeda, Shoko
• Granted Patent: US 9,672,432 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

An image generation device that enhances visual recognizability between a recognized 3D (three-dimensional) object and a substitute image synthesized in an area of a 3D object image which is a photographed image of the 3D object includes an image synthesis section that recognizes a 3D object present in the peripheral area of a vehicle and outputs 3D object attribute information indicative of attributes of this 3D object, determines a 3D object image area as an image area of the 3D object in the photographed image based on position information included in the 3D object attribute information, outputs at least one of a substitute image of the 3D object applied with a color based on the color information and a substitute image of the 3D object under a specified directional posture of the 3D object specified based on type information and directional posture information and generates a bird'"'"'s-eye view image with a substitution image in which the substitution image outputted from a substitution image output section is synthesized at the position of the 3D object image area.

Claims

1. An image generation device configured to output, as a displaying image, a bird'"'"'s-eye view image generated by projection transformation at an upper virtual viewpoint of a photographed image obtained by an in-vehicle camera for photographing a peripheral area of a vehicle, the image generation device comprising:

a 3D object recognition section for recognizing a 3D object present in the peripheral area and outputting 3D object attribute information representing attribute of the 3D object;

a 3D object image area determination section for determining a 3D object image area which is an image area of the 3D object in the photographed image based on position information included in the 3D object attribute information;

a substitute image output section for outputting at least one of a substitute image of the 3D object colored with a color based on color information included in the 3D object attribute information and a substitute image of the 3D object with a directional posture identified for the 3D object identified based on type information and directional posture information included in the 3D object attribute information; and

an image synthesis section for generating a bird'"'"'s-eye view image synthesized and attached with a substitute image outputted from the substitute image output section at a position in the 3D object image area.

2. The image generation device according to claim 1, wherein the substitute image output section includes:

a substitute image database storing substitute images of 3D objects based on color information;

a substitute image retrieval section for retrieving, from the substitute image database, a substitute image of the 3D object based on 3D object type information included in the 3D object attribute information; and

a substitute image coloration section for coloring a substitute image retrieved by the substitute image retrieval section with a color based on color information included in the 3D object attribute information.

3. The image generation device according to claim 2, wherein the substitute image coloration section determines a plurality of colors for characterizing the 3D object based on the color information and paints the substitute image with the plurality of colors separately.

4. The image generation device according to claim 2, wherein the device further comprises a color enhancement instruction section for outputting a color enhancement instruction for enhancing the color to be added to the substitute image by the substitute image coloration section, based on the 3D object attribute information.

5. The image generation device according to claim 1, wherein the device further comprises a bird'"'"'s-eye view photographed image generation section for generating a bird'"'"'s-eye view photographed image of the vehicle periphery from a plurality of photographed images covering the vehicle periphery and having different photographing directions with using projection transformation, and the image synthesis section apples the substitute image to the 3D object image area of the bird'"'"'s-eye view photographed image.

6. The image generation device according to claim 1, wherein in addition to the substitute image, there is provided an image showing the moving direction of the 3D object.

7. The image generation device according to claim 6, wherein the image showing the moving direction of the 3D object is an after-image of the substitute image corresponding to the movement of the 3D object.

8. The image generation device according to claim 6, wherein the image showing the moving direction of the 3D object is an arrow.

9. The image generation device according to claim 1, wherein the substitute image output section includes:

a substitute image database storing substitute images of 3D objects based on type information and directional posture information; and

a substitute image retrieval section for retrieving, from the substitute image database, the substitute image with using the type information and the directional posture information included in the 3D object attribute information as a search condition.

10. The image generation device according to claim 1, wherein the 3D object recognition section includes a face detection section for detecting a human face, and based on detection result by the face detection section, the recognized 3D object comprising a human being set as the type information and one of the forward-facing directional posture, a rearward-facing directional posture and a lateral facing directional posture being set as the directional posture information.

11. The image generation device according to claim 1, wherein in the 3D object recognition section, based on inter-vehicle communication information obtained by inter-vehicle communication, a recognized 3D object comprising an automobile is set as the type information, and the automobile assuming one of the forward facing directional posture, the rearward facing directional posture and the lateral facing directional posture is set as the directional posture information.

12. The image generation device according to claim 1, wherein the device further comprises a bird'"'"'s-eye view photographed image generation section for generating a bird'"'"'s-eye view photographed image of the vehicle periphery from a plurality of photographed images covering the periphery of the vehicle and having different photographing directions, with using projection transformation;

and the image synthesis section applies the substitute image to the 3D object image area of the bird'"'"'s-eye view photographed image.