Image processing device and method for parking support

• US 7,734,417 B2
• Filed: 05/19/2006
• Issued: 06/08/2010
• Est. Priority Date: 05/20/2005
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Yanai, Tatsumi, Chinomi, Satoshi
• Primary Examiner(s): Black; Thomas G
• Assistant Examiner(s): Chen; Shelley
• Application Number:
  US11/438,117
• Publication Number:
  US 20070003108A1
• Time in Patent Office: 1,481 Days
• Field of Search: 340/932.2, 701/301, 348/118
• US Class Current: 701/301
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/102 using 360 degree surveillance camera system
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/70 characterised by an event-triggered choice to display a specific image among a selection of captured images
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G01S 15/86 Combinations of sonar systems with lidar systems; Combinations of sonar systems with systems not using wave reflection
• CPC Class Codes: G01S 15/931 of land vehicles
• CPC Class Codes: G01S 2013/9314 Parking operations
• CPC Class Codes: G01S 2015/935 for measuring the contour, e.g. a trajectory of measurement points, representing the boundary of the parking space
• CPC Class Codes: G03B 37/06 involving anamorphosis G03B37/02, G03B37/04 take precedence

Abstract

An image processing device and method for parking support that allows the driver to perceive the state surrounding the vehicle correctly in a simple operation even if plural cameras are carried on the vehicle. Plural images are acquired using plural cameras for image pickup in different directions surrounding the vehicle. These images are used to show the state surrounding the vehicle to the driver, preferably in the form of a bird'"'"'s-eye view. The view seen by the driver can switch from the bird'"'"'s-eye view to an image of a desired view. This view can be selected by the driver by, for example, touching the screen upon which the bird'"'"'s-eye view is displayed.

Claims

1. An image display device for a vehicle, comprising:

plural cameras for capturing images in different directions surrounding the vehicle;

an image processing device operable to use plural images acquired with the plural cameras to form a bird'"'"'s-eye view showing the entire view surrounding the vehicle;

a display device operable to provide the bird'"'"'s-eye view to a driver of the vehicle;

a selector operable to allow selection of a desired view from the vehicle;

wherein the image processing device is operable to switch from the bird'"'"'s-eye view to an image corresponding to the desired view when the desired view is selected by the selector; and

an obstacle detector operable to generate a signal indicative of an obstacle approaching the vehicle;

wherein the image processing device is operable to switch to the bird'"'"'s-eye view from the image corresponding to the desired view when the obstacle detector indicates a first obstacle approaching the vehicle;

wherein the image processing device is operable to switch from the bird'"'"'s-eye view to an obstacle image in a direction of the first obstacle in response to indication of the first obstacle by the obstacle detector; and

wherein the image processing device is operable to switch from the obstacle image in the direction of the first obstacle to the bird'"'"'s-eye view in response to indication of a second obstacle by the obstacle detector.

2. The image display device according to claim 1 wherein the selector is operable by the driver to select the desired view the driver wishes to watch while the bird'"'"'s-eye view is displayed.

3. The image display device according to claim 1,

wherein the image processing device is operable to switch to the bird'"'"'s-eye view from the image corresponding to the desired view when the obstacle detector indicates a first obstacle approaching the vehicle while the image corresponding to the desired view is displayed.

4. The image display device according to claim 1, further comprising:

a visible indicator for indicating a presence of the first obstacle and a presence of the second obstacle in the bird'"'"'s-eye view.

5. The image display device according to claim 3, further comprising:

a visible indicator for indicating a presence of the first obstacle and a presence of a second obstacle in the bird'"'"'s-eye view when the obstacle detector indicates the second obstacle approaching the vehicle.

6. The image display device according to claim 3 wherein the display device comprises a touch-panel type touch coordinate detecting mechanism;

and wherein the selector further comprises a screen of the touch-panel type touch coordinate detecting mechanism, the screen operable for manipulation using contact by the driver.

7. The image display device according to claim 1 wherein the display device comprises a touch-panel type touch coordinate detecting mechanism;

and wherein the selector further comprises a screen of the touch-panel type touch coordinate detecting mechanism, the screen operable for manipulation using contact by the driver.

8. The image display device according to claim 7, further comprising:

plural regions dividing the bird'"'"'s-eye view provided on the display device; and

wherein the display device is further operable to display the image corresponding to the desired view in a region corresponding to desired view.

9. An image display device for a vehicle, comprising:

image pickup means for taking plural images in plural different directions surrounding the vehicle;

image processing means for forming a bird'"'"'s-eye view showing the entire view surrounding the vehicle using the plural images;

display means for showing the bird'"'"'s-eye view;

switching means for switching from the bird'"'"'s-eye view to a view from one of the image pickup means, the view selectable by the driver when the bird'"'"'s-eye view is displayed on the display means, means for detecting an obstacle approaching the vehicle;

means for switching from the bird'"'"'s-eye view to an obstacle image in a direction of the first obstacle in response to indication of the first obstacle by the obstacle detector; and

means for switching from the obstacle image in the direction of the first obstacle to the bird'"'"'s-eye view in response to indication of a second obstacle by the obstacle detector.

10. The image display device according to claim 9, further comprising:

means for switching to the bird'"'"'s-eye view from the view from one of the image pickup means when the obstacle first approaching the vehicle is detected and the view from one of the image pickup means is displayed on the display means.

11. The image display device according to claim 9, further comprising:

means for dividing the bird'"'"'s-eye view shown on the display means into plural regions surrounding the vehicle; and

wherein the display means displays the view as an image in one of the plural regions in a direction corresponding to the view.

12. An image display method for a vehicle, comprising:

capturing plural images surrounding the vehicle in plural different directions;

forming a bird'"'"'s-eye view illustrating the entirety of the view surrounding the vehicle using the plural images;

displaying the bird'"'"'s-eye view;

selecting a desired view captured by the plural images;

switching from the bird'"'"'s-eye view to the desired view,

detecting a first obstacle approaching the vehicle;

switching to the bird'"'"'s-eye view from the desired view when the first obstacle approaching the vehicle is detected and the desired view is displayed;

switching to an obstacle image in a direction of the first obstacle when the obstacle approaching the vehicle continues to be detected; and

switching from the obstacle image in the direction of the first obstacle to the bird'"'"'s-eye view in response to detecting a second obstacle approaching the vehicle.

13. The image display method according to claim 12, further comprising:

indicating, in the bird'"'"'s-eye view, a presence of the first obstacle and a presence of the second obstacle approaching the vehicle.

14. The image display method according to claim 12, further comprising:

selecting the desired view using a screen displaying the bird'"'"'s-eye view.

15. The image display method according to claim 14, further comprising:

dividing the bird'"'"'s-eye view into plural regions surrounding the vehicle; and

displaying the desired view as an image in one of the plural regions in a direction corresponding to the desired view.