Image display device and method

• US 7,592,928 B2
• Filed: 06/06/2006
• Issued: 09/22/2009
• Est. Priority Date: 06/07/2005
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Oizumi, Ken, Kawano, Tsutomu, Takano, Teruhisa, Seguchi, Hidenori, Chinomi, Satoshi, Yanagi, Takura
• Primary Examiner(s): Wu; Daniel
• Assistant Examiner(s): Lu; Shirley
• Application Number:
  US11/447,435
• Publication Number:
  US 20060274147A1
• Time in Patent Office: 1,204 Days
• Field of Search: 340/932.2, 340/435, 340/436, 340/932.3, 348143-170
• US Class Current: 340/932.2
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/102 using 360 degree surveillance camera system
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/70 characterised by an event-triggered choice to display a specific image among a selection of captured images
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G01S 2013/9314 Parking operations
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A device and method that allows any differences between the relative position of an obstacle as it appears in a bird'"'"'s-eye view image and its actual relative position can be checked on the same screen. In order to display the vehicle surroundings to the driver, multiple direct images obtained from multiple pickup devices, preferably cameras, oriented in different directions are used to compose a bird'"'"'s-eye view image from which the vehicle surroundings can be known at a glance. The bird'"'"'s-eye view image and a direct image picked up in the direction of an obstacle with respect to the vehicle are simultaneously displayed when the obstacle is detected in the vicinity of the vehicle.

Claims

1. An image display device for a vehicle, comprising:

multiple image pickup devices that capture multiple direct images of the vehicle surroundings in different directions;

an image-processing device that composes a bird'"'"'s-eye view image using the multiple direct images; and

a display device that provides at least one of the bird'"'"'s-eye view image and a direct image picked up by one of the multiple image pickup devices to the driver; and

wherein the image-processing device displays simultaneously on the display device at least two discrete images, a first of the at least two discrete images including the bird'"'"'s-eye view image and a second of the at least two discrete images including a direct image picked up by only one of the multiple image pickup devices;

wherein the image-processing device displays the second of the at least two discrete images with a display size larger than a display size of the first of the at least two discrete images; and

wherein the image-processing device selects the second discrete image to be displayed in response to at least one of a position of a detected obstacle and a gear change of the vehicle.

2. The image display device according to claim 1 wherein the display device displays an obstacle alarm in the first discrete image to emphasize a portion of the vehicle approaching the obstacle.

3. The image display device according to claim 2 wherein the obstacle alarm is the second discrete image superimposed over a portion of the first discrete image including the vehicle approaching the obstacle in the bird'"'"'s-eye view image.

4. The image display device according to claim 1, further comprising:

an obstacle detector that detects the obstacle; and

at least one guideline indicating a positional relationship of the vehicle to vehicle surroundings, the at least one guideline superimposed on at least one of the first discrete image and the second discrete image and arranged to indicate a direction of travel of the vehicle.

5. The image display device according to claim 4 wherein the at least one guideline has a first display state;

and wherein the display device changes the first display state to a second display state of at least a portion of the at least one guideline when the obstacle is detected by the obstacle detector.

6. The image display device according to claim 4 wherein the at least one guideline has a first display state;

and wherein the display device erases a portion of the at least one guideline closest to the obstacle when the obstacle is detected by the obstacle detector.

7. The image display device according to claim 4 wherein the display device flashes a portion of the at least one guideline closest to the obstacle when the obstacle is detected by the obstacle detector.

8. The image display device according to claim 4 wherein the display device changes a color of a portion of the at least one guideline closest to the obstacle when the obstacle is detected by the obstacle detector.

9. The image display device according to claim 1 wherein the display device emphasizes a portion of the second discrete image closest the obstacle.

10. The image display device according to claim 9, further comprising:

a line enclosing and emphasizing the portion of the second discrete image closest the obstacle.

11. The image display device according to claim 1, further comprising:

a first background color external of at least one of the first and second discrete images; and

wherein the display device changes the first background color to a second background color.

12. An image display device for a vehicle, comprising:

means for capturing vehicle surroundings from a plurality of directions;

means for composing a bird'"'"'s-eye view image from multiple images of the vehicle surroundings so captured;

means for detecting a presence of any obstacles in the vicinity of the vehicle;

means for simultaneously displaying at least two discrete images to the driver, a first of the at least two discrete images including the bird'"'"'s-eye view image and a second of the at least two discrete images including a single direct image picked up in a direction of an obstacle in response to detection of the obstacle; and

means for sizing the second of the at least two discrete images to have a larger display size than a display size of the first of the at least two discrete images.

13. An image display method for a vehicle, comprising:

capturing vehicle surroundings from a plurality of directions;

composing a bird'"'"'s-eye view image from multiple images of the vehicle surroundings so captured;

detecting a presence of any obstacles in the vicinity of the vehicle;

simultaneously displaying at least two discrete images to the driver, a first of the at least two discrete images including the bird'"'"'s-eye view image and a second of the at least two discrete images including a single direct image picked up in a direction of an obstacle in response to detection of the obstacle; and

sizing the second of the at least two discrete images to have a larger display size than a display size of the first of the at least two discrete images.

14. The image display method according to claim 13, further comprising:

superimposing guidelines indicating a positional relationship between the vehicle and vehicle surroundings and indicating a direction of travel of the vehicle on at least one of the first and second discrete images.

15. The image display method according to claim 14, further comprising:

changing a display state of the guidelines when the obstacle is detected.

16. The image display method according to claim 15 wherein changing the display state of the guidelines when the obstacle is detected further comprises at least one of erasing a portion of the guidelines closest to the obstacle when the obstacle is detected, flashing the portion of the guidelines closest to the obstacle when the obstacle is detected, and changing the color of a portion of the guidelines closest to the obstacle when the obstacle is detected.

17. The image display method according to claim 13, further comprising:

emphasizing at least a portion of the vehicle approaching the obstacle on at least one of the bird'"'"'s-eye view image and the direct image.

18. The image display method according to claim 17 wherein emphasizing at least the portion of the vehicle approaching the obstacle further comprises at least one of concealing the portion of the vehicle approaching the obstacle on the first discrete image using the second discrete image, enclosing the portion of the second discrete image closest the obstacle with a line, making an area of a display device occupied by the second discrete image larger than an area of a display device occupied by the first discrete image, and changing a color of a background bordering at least one of the two discrete images.

19. An image display device for a vehicle, comprising:

multiple image pickup devices that capture multiple direct images of the vehicle surroundings in different directions;

an image-processing device that composes a bird'"'"'s-eye view image using the multiple direct images; and

a display device that provides the bird'"'"'s-eye view image to the driver;

wherein the image-processing device displays simultaneously on the display device at least two discrete images, a first of the at least two discrete images including the bird'"'"'s-eye view image and a second of the at least two discrete images including a direct image picked up by only one of the multiple direct image pickup devices; and

wherein the display size of the second of the at least two discrete images is larger than the display size of the first of the at least two discrete images.

20. The image display device of claim 19 wherein the image-processing device selects the second discrete image as a front image ahead of the vehicle when a shift position of the vehicle is configured to propel the vehicle forward;

and wherein the image-processing device selects the second discrete image as a rear image behind the vehicle when a shift position of the vehicle is configured to propel the vehicle rearward.

21. The image display device of claim 19 wherein the image-processing device selects the second discrete image based on the vehicle speed.