Image Processing Device

• US 20080198226A1
• Filed: 11/19/2007
• Published: 08/21/2008
• Est. Priority Date: 02/21/2007
• Status: Active Grant

• Current Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.)
• Original Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.)
• Inventors: Imamura, Kosuke
• Granted Patent: US 8,330,816 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

The present invention provides an image processing device capable of enabling accurate recognition of a solid object present near a vehicle and displaying the solid object. The image processing device includes a viewpoint conversion unit for receiving data of images captured by at least one image capturing camera and generating a top view image, a solid object extraction unit for detecting a solid object from the data of the images captured by the at least one image capturing camera and extracting the solid object, a solid object image generation unit for generating a solid object image in accordance with the solid object extracted by the solid object extraction unit, and an image synthesis unit for synthesizing the solid object image generated by the solid object image generation unit with the top view image generated by the viewpoint conversion unit.

Claims

1. An image processing device configured for displaying on a display an image of surroundings of a vehicle, the image processing device comprising:

at least one image capturing camera configured for capturing images of the surroundings of the vehicle and producing data of the captured images;

a viewpoint conversion unit configured for performing viewpoint conversion on the data of the captured images to convert the data into an image looking down from a point of view above the vehicle;

a solid object extraction unit configured for extracting a solid object which has a relative speed with respect to the vehicle included in the data of the captured images or with respect to a surrounding environment;

a solid object image generation unit configured for generating a solid object image in accordance with the extracted solid object;

an image synthesis unit configured for synthesizing the generated solid object image with the viewpoint-converted image; and

a display control unit configured for displaying on the display the image synthesized by the image synthesis unit.

2. The image processing device according to claim 1,

wherein the solid object extraction unit further extracts the outline of the solid object, and

wherein the solid object image generation unit generates the solid object image in accordance with the extracted outline of the solid object.

3. The image processing device according to claim 1,

wherein the solid object image generation unit generates one of a solid object image directly showing the extracted solid object, a solid object image emphasizing the outline of the solid object, and a solid object image with a shadow.

4. The image processing device according to claim 1,

wherein the image synthesis unit determines a synthesis position of the solid object image on the basis of a particular solid object included in the viewpoint-converted image.

5. The image processing device according to claim 1,

wherein the at least one image capturing camera includes a plurality of image capturing cameras for capturing images of a front side, a rear side, and lateral sides of the vehicle.

6. The image processing device according to claim 1,

wherein the solid object extraction unit further extracts at least one of the moving direction, the size, and the distance from the vehicle of the solid object, and

wherein the solid object image generation unit generates the solid object image in accordance with the extracted at least one of the moving direction, the size, and the distance from the vehicle of the solid object.

7. The image processing device according to claim 6,

wherein the image synthesis unit determines the direction and the size of the solid object image in accordance with at least one of the moving direction, the size, and the distance from the vehicle of the solid object extracted by the solid object extraction unit.

8. The image processing device according to claim 1,

wherein the image synthesis unit determines the color of the solid object image to distinguish the color from the color of a surrounding image.

9. The image processing device according to claim 8,

wherein a color having a large contrast difference from the color of the surrounding image is determined as the color of the solid object image.

10. A navigation system comprising:

a navigation device; and

an image processing device configured for displaying on a display an image of surroundings of a vehicle, the image processing device including

at least one image capturing camera configured for capturing images of the surroundings of the vehicle and producing data of the captured images,

a viewpoint conversion unit configured for performing viewpoint conversion on the data of the captured images to convert the data into an image looking down from a point of view above the vehicle,

a solid object extraction unit configured for extracting a solid object which has a relative speed with respect to the vehicle included in the data of the captured images or with respect to a surrounding environment,

a solid object image generation configured unit for generating a solid object image in accordance with the extracted solid object,

an image synthesis unit configured for synthesizing the generated solid object image with the viewpoint-converted image, and

a display control unit configured for displaying on the display the image synthesized by the image synthesis unit.

11. The navigation system according to claim 10,

wherein the navigation device includes a control unit for determining at least one of whether or not the vehicle is stopped and whether or not the vehicle is running at a predetermined low speed, and

wherein the display control unit displays the synthesized image when the control unit determines that the vehicle is stopped or running at the low speed.

12. A computer readable medium storing an image processing program configured for displaying on a display an image of surroundings of a vehicle, the image processing program comprising:

a step of capturing images of the surroundings of the vehicle and extracting a solid object which has a relative speed with respect to the vehicle included in data of the captured images or with respect to a surrounding environment;

a step of generating a solid object image in accordance with the extracted solid object;

a step of performing viewpoint conversion on the data of the captured images to convert the data into an image looking down from a point of view above the vehicle;

a step of synthesizing the solid object image with the viewpoint-converted image; and

a step of displaying the synthesized image on the display.

13. The image processing program according to claim 12,

wherein the extracting step includes extracting the outline of the solid object, and

wherein the generating step includes generating the solid object image in accordance with the extracted outline of the solid object.

14. The image processing program according to claim 12,

wherein the generating step includes generating one of a solid object image directly showing the extracted solid object, a solid object image emphasizing the outline of the solid object, and a solid object image with a shadow.

15. The image processing program according to claim 12,

wherein the synthesizing step includes determining the color of the solid object image to distinguish the color from the color of a surrounding image.

16. The image processing program according to claim 12,

wherein the synthesizing step includes determining a synthesis position of the solid object image on the basis of a particular solid object included in the viewpoint-converted image.

17. The image processing program according to claim 12,

wherein the viewpoint-converting step includes performing the viewpoint conversion on the data of the captured images of a front side, a rear side, and lateral sides of the vehicle to generate omnidirectional data of the captured images centering on the vehicle.

18. The image processing program according to claim 12,

wherein the extracting step includes extracting at least one of the moving direction, the size, and the distance from the vehicle of the solid object, and

wherein the generating step includes generating the solid object image in accordance with the extracted at least one of the moving direction, the size, and the distance from the vehicle of the solid object.

19. The image processing program according to claim 18,

wherein the synthesizing step includes determining the direction of the solid object image in accordance with the extracted moving direction of the solid object.

20. An image display method for displaying on a display an image of surroundings of a vehicle, the image display method comprising:

capturing at least one image of the surroundings of the vehicle and extracting a solid object included in data of the at least one captured image;

generating a solid object image in accordance with the extracted solid object;

performing viewpoint conversion on the data of the at least one captured image to convert the data into an image looking down from a point of view above the vehicle;

synthesizing the solid object image with the viewpoint-converted image; and

displaying the synthesized image on the display.