DEVICE AND METHOD FOR MONITORING VEHICLE SURROUNDINGS

• US 20100220189A1
• Filed: 07/31/2006
• Published: 09/02/2010
• Est. Priority Date: 08/02/2005
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Yanagi, Takura
• Granted Patent: US 8,885,045 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 19/483 with obstacle sensors of electric or electronic type
• CPC Class Codes: B60R 2300/102 using 360 degree surveillance camera system
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/70 characterised by an event-triggered choice to display a specific image among a selection of captured images
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8093 for obstacle warning

Abstract

A vehicle surrounding monitoring device include at least one camera 2 installed in an own vehicle to image a video around the own vehicle, an obstacle sensor 3 for detecting an obstacle within an imaging range of the camera 2, a pixel synthesis unit 13 for receiving a camera image imaged by the camera 2 and converting the camera image into a view point converted image seen from a virtual view point above the own vehicle, and a display device 4 for displaying the view point conversion image converted by the pixel synthesis unit 13. Simultaneously when a warning is given by a warning sound upon entry of an obstacle within an obstacle detection range of the obstacle sensor 3, an image of the obstacle detection range is synthesized with an obstacle image imaged by the camera 2 and included in the view point converted image to be in contact therewith.

Claims

1. A vehicle surrounding monitoring device, comprising:

at least one camera installed in an own vehicle to obtain an image of around the own vehicle;

an obstacle detection device installed in the own vehicle configured to detect an obstacle in an obstacle detection area;

an image processing device configured to convert a camera image imaged by the camera into a view point converted image seen from a virtual view point above the own vehicle; and

a display configured to display the view point converted image of the image processing device,

wherein, simultaneously when a warning is given upon entry of an obstacle within an obstacle detection range of the obstacle detection device, the image processing device modifies at least one of an image of the obstacle detection range of the obstacle detection device and an obstacle image imaged by the camera and included in the view point converted image so that the image of the obstacle detection range of the obstacle detection device and the obstacle image substantially contact one another.

2. The vehicle surrounding monitoring device according to claim 1, wherein the image processing device is configured to set an image obtained by projecting the obstacle detection range of the obstacle detection device on a ground surface, with an attaching position of the camera set as a view point, as the image of the obstacle detection range, and is configured to modify the image of the obstacle detection range projected on the ground surface with the view point converted image seen from the virtual view point above the own vehicle.

3. The vehicle surrounding monitoring device according to claim 1, wherein the image processing device sterically modifies the image of the obstacle detection range of the obstacle detection device with the view point converted image.

4. The vehicle surrounding monitoring device according to claim 3, wherein the image of the obstacle detection range comprises a stereoscopic image that includes an image of an area where the obstacle detection range is projected on a ground surface, and a plane extending vertically from a boundary of an area where the obstacle detection range of the obstacle detection device is projected on the ground surface with an attaching position of the camera set as a view point, and is modified as the view point converted image to be displayed.

5. The vehicle surrounding monitoring device according to claim 1, wherein the obstacle detection device is configured to detect an obstacle in space away from a ground surface;

and

wherein an image of an area where the obstacle detection range is projected on the ground surface, and an image displayed in a position where the obstacle detected by the obstacle detection device is projected on the ground surface and brought into contact with the image of the area where the obstacle detection range is projected on the ground surface are modified as the view point converted image to be displayed.

6. The vehicle surrounding monitoring device according to claim 1, wherein the image processing device is configured to display the image of the obstacle detection range in a semitransparent manner.

8. The vehicle surrounding monitoring device according to claim 1, wherein the warning is provided by a sound.

7. A vehicle surrounding monitoring method, comprising:

obtaining an image around an own vehicle by at least one camera installed in the own vehicle;

detecting an obstacle within an imaging range of the camera by an obstacle detection sensor; and

when presence of an obstacle is detected by the obstacle detection sensor, giving a warning, and simultaneously modifying at least one of an image of an obstacle detection range of the obstacle detection sensor and an image of an obstacle imaged by the camera to be in contact with one another to be included in a view point converted image.

9. A vehicle surrounding monitoring device, comprising:

at least one camera installed in an own vehicle to obtain an image around the own vehicle;

obstacle detection means installed in the own vehicle for detecting an obstacle in an obstacle detection area;

image processing means for converting a camera image imaged by the camera into a view point converted image seen from a virtual view point above the own vehicle; and

display means for displaying the view point converted image of the image processing means,

wherein, simultaneously when a warning is given upon entry of an obstacle within an obstacle detection range of the obstacle detection means, the image processing means modifies at least one of an image of the obstacle detection range of the obstacle detection means and an obstacle image imaged by the camera and included in the view point converted image so that the image of the obstacle detection range of the obstacle detection means and the obstacle image contact one another.

10. A computer readable medium having program code recorded therein, that, when executed on a computing system, causes the computing system to perform steps comprising:

obtaining an image around an own vehicle by at least one camera installed in the own vehicle;

detecting an obstacle within an imaging range of the camera by an obstacle detection sensor; and

when presence of an obstacle is detected by the obstacle detection sensor, giving a warning, and simultaneously modifying at least one of an image of an obstacle detection range of the obstacle detection sensor and an image of an obstacle imaged by the camera to be in contact with one another to be included in a view point converted image.