Driving support system and method of producing overhead view image

• US 7,684,593 B2
• Filed: 10/24/2005
• Issued: 03/23/2010
• Est. Priority Date: 10/25/2004
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Chinomi, Satoshi
• Primary Examiner(s): Ahmed; Samir A.
• Assistant Examiner(s): Fitzpatrick; Atiba O
• Application Number:
  US11/256,533
• Publication Number:
  US 20060088190A1
• Time in Patent Office: 1,611 Days
• Field of Search: 382103-107, 340907-950, 348112-120, 345619-689
• US Class Current: 382/104
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A driving support system which includes: at least two cameras to take images of areas around a vehicle, whose ranges of field of view partially overlap with each other; a control unit performing a predetermined coordinate transformation on the images taken by the cameras and joining the transformed images together to produce an overhead view image; and a display unit to display the overhead view image. The control unit changes the position of the joint of the transformed images in the overhead view image.

Claims

1. A driving support system comprising:

a plurality of image taking devices configured to take images of areas around a vehicle, wherein the image taking devices have ranges of field of view that partially overlap with each other;

an image processing unit configured to perform a predetermined coordinate transformation on the images taken by the image taking devices to produce transformed images and to join the transformed images to each other at a joint to produce a virtual image viewed from a virtual viewpoint, wherein the image processing unit is configured to continuously change a position of the joint relative to the transformed images in the virtual image and to stop changing the position of the joint, based on pixel data of the produced virtual image; and

a display unit configured to display the virtual image.

2. The driving support system according to claim 1, wherein the image processing unit is configured to continuously change the position of the joint independently of a driver'"'"'s operation.

8. The driving support system according to claim 1, wherein, when the virtual image includes a plurality of joints, the image processing unit is configured to change the position of only a part of the joints.

9. The driving support system according to claim 8, further comprising a determining unit configured to determine a direction of movement of the vehicle in a longitudinal direction thereof by receiving a gear range signal indicating a selected gear range, wherein the image processing unit is configured to change the position of the joint lying forward in the direction of movement determined by the determining unit.

10. The driving support system according to claim 8, further comprising a determining unit configured to determine a direction of movement of the vehicle in a transverse direction thereof by receiving a steering signal indicating a steering-wheel operation, wherein the image processing unit is configured to change the position of the joint lying forward in the direction of movement determined by the determining unit.

3. A driving support system, comprising:

a plurality of image taking devices configured to take images of areas around a vehicle, wherein the image taking devices have ranges of field of view that partially overlap with each other;

an image processing unit configured to perform a predetermined coordinate transformation on the images taken by the image taking devices to produce transformed images and to join the transformed images to each other at a joint to produce a virtual image viewed from a virtual viewpoint, wherein the image processing unit is configured to continuously change a position of the joint relative to the transformed images in the virtual image; and

a display unit configured to display the virtual image,

wherein the image processing unit is configured to continuously change the position of the joint while the vehicle is not in a reverse mode, and to stop changing the position of the joint while the vehicle is in the reverse mode.

4. The driving support system according to claim 3, further comprising a determining unit configured to determine whether the vehicle is being reversed by detecting a driver'"'"'s operation or vehicle behavior, wherein the image processing unit is configured to stop changing the position of the joint when the determining unit determines that the vehicle is in the reverse mode.

5. The driving support system according to claim 3, wherein the image processing unit is configured to determine, based on pixel data of the produced virtual image, whether to stop changing the position of the joint, and to stop changing the position of the joint while a determination of whether to stop is being made even while the vehicle is not in the reverse mode.

6. The driving support system according to claim 5, wherein the image processing unit is configured to determine modes of color values of pixels in predetermined areas of the virtual image, which are provided on both sides of the joint and bordered by the joint, while changing the position of the joint, and to determine to stop changing the position of the joint at a moment when the modes of color values of the pixels in respective areas become different from each other.

7. A driving support system, comprising:

a plurality of image taking devices configured to take images of areas around a vehicle, wherein the image taking devices have ranges of field of view that partially overlap with each other;

an image processing unit configured to perform a predetermined coordinate transformation on the images taken by the image taking devices to produce transformed images and to join the transformed images to each other at a joint to produce a virtual image viewed from a virtual viewpoint, wherein the image processing unit is configured to change a position of the joint in the virtual image; and

a display unit configured to display the virtual image;

wherein the image processing unit is configured to continuously change the position of the joint while the vehicle is not in a reverse mode, and to stop changing the position of the joint while the vehicle is in the reverse mode;

wherein the image processing unit is configured to determine, based on pixel data of the produced virtual image, whether to stop changing the position of the joint, and to stop changing the position of the joint while a determination of whether to stop is being made even while the vehicle is not in the reverse mode;

wherein the image processing unit is configured to determine mode of color values of pixels in a predetermined area of the virtual image bordered by the joint, and to compute percentage of pixels with color value of the mode of color values with respect to a total number of pixels in the predetermined area, while changing the position of the joint, and to determine to stop changing the position of the joint at a moment when the percentage stops increasing.

11. A method of producing a virtual image to be displayed in a driving support system, comprising:

taking images of areas around a vehicle partially overlapping with each other;

performing a predetermined coordinate transformation on the taken images to produce transformed images;

joining the transformed images together at a joint to produce a virtual image viewed from a virtual viewpoint; and

continuously changing a position of the joint relative to the transformed images in the virtual image and stopping changing the position of the joint, based on pixel data of the produced virtual image.

12. A driving support system comprising:

a plurality of image taking means for taking images of areas around a vehicle, wherein the image taking means have ranges of field of view that partially overlap with each other;

image processing means for performing a predetermined coordinate transformation on the images taken by the image taking means to produce transformed images and joining the transformed images to each other at a joint to produce a virtual image viewed from a virtual viewpoint, wherein the image processing means continuously changes a position of the joint relative to the transformed images in the virtual image and stops changing the position of the joint, based on pixel data of the produced virtual image; and

display means for displaying the virtual image.

13. A driving support system, comprising:

a plurality of image taking devices configured to take images of areas around a vehicle, wherein the image taking devices have ranges of field of view that partially overlap with each other;

an image processing unit configured to perform a predetermined coordinate transformation on the images taken by the image taking devices to produce transformed images and to join the transformed images to each other at a joint to produce a virtual image viewed from a virtual viewpoint, wherein the image processing unit is configured to continuously change a position of the joint relative to the transformed images in the virtual image; and

a display unit configured to display the virtual image;

wherein the position of the joint relative to the transformed images in the virtual image is changed in a continuous manner without a driver'"'"'s operation to initiate the continuous change of the joint position.

14. A method of producing a virtual image to be displayed in a driving support system, comprising:

taking images of areas around a vehicle partially overlapping with each other;

performing a predetermined coordinate transformation on the taken images to produce transformed images;

joining the transformed images together at a joint to produce a virtual image viewed from a virtual viewpoint;

continuously changing a position of the joint relative to the transformed images in the virtual image; and

changing position of the joint relative to the transformed images in the virtual image in a continuous manner without a driver'"'"'s operation to initiate the continuous change of the joint position.

15. A driving support system comprising:

a plurality of image taking means for taking images of areas around a vehicle, wherein the image taking means have ranges of field of view that partially overlap with each other;

image processing means for performing a predetermined coordinate transformation on the images taken by the image taking means to produce transformed images and joining the transformed images to each other at a joint to produce a virtual image viewed from a virtual viewpoint, wherein the image processing means continuously changes a position of the joint relative to the transformed images in the virtual image; and

display means for displaying the virtual image;

wherein the image processing means is configured to change position of the joint relative to the transformed images in the virtual image in a continuous manner without a driver'"'"'s operation to initiate the continuous change of the joint position.