Rendering device for generating a drive assistant image for drive assistance

• US 6,369,701 B1
• Filed: 06/28/2001
• Issued: 04/09/2002
• Est. Priority Date: 06/30/2000
• Status: Expired due to Term

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Yoshida, Takashi, Iisaka, Atsushi, Ishida, Akira, Yasui, Nobuhiko
• Primary Examiner(s): Wu, Daniel J.
• Assistant Examiner(s): Nguyen, Tai T.
• Application Number:
  US09/892,740
• Time in Patent Office: 285 Days
• Field of Search: 340/435, 340/461, 340/903, 340/904, 340/436, 340/438, 340/439, 348/148, 348/61, 348/218, 382/104, 382/282, 382/293
• US Class Current: 340/435
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/107 using stereoscopic cameras
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/60 characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8086 for vehicle path indication
• CPC Class Codes: B62D 15/0275 by overlaying a vehicle path based on present steering angle over an image without processing that image
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 5/80 Geometric correction

Abstract

In a rendering device Urnd, a CPU generates a lower viewpoint image based on captured images captured by image capture devices both securely mounted to a vehicle. Here, the lower viewpoint image shows a region rear of the vehicle viewed from those image capture devices. Then, the CPU executes viewpoint conversion processing onto the captured images, and generates an upper viewpoint image. Here, the upper viewpoint image shows also the region rear of the vehicle but viewed from a camera virtually placed above the vehicle. The CPU then stitches the upper and lower viewpoint images together to generate a drive assistant image. In such manner, the drive assistant image generated by the rendering device Urnd becomes informative enough for a driver of the vehicle.

Claims

1. A rendering device for generating a drive assistant image for drive assistance, comprising:

a reception part for receiving an image captured by an image capture device placed in a vehicle;

a first image generating part for generating, based on the captured image received by said reception part, a lower viewpoint image showing a surrounding region of said vehicle viewed from a first viewpoint;

a second image generating part for generating, based on the captured image received by said reception part, an upper viewpoint image showing said surrounding region viewed from a second viewpoint differing from said first viewpoint; and

a third image generating part for generating the drive assistant image including the lower viewpoint image generated by said first image generating part, and the upper viewpoint image generated by said second image generating part;

wherein

a first angle formed between a road surface and a direction of said first viewpoint to said surrounding region is smaller than a second angle formed between the road surface and a direction of said second viewpoint to said surrounding region.

2. The rendering device according to claim 1, wherein

said first image generating part generates the lower viewpoint image showing a first range in said surrounding region viewed from said first viewpoint,

said second image generating part generates the upper viewpoint image showing a second range in said surrounding region viewed from said second viewpoint, and

said second range is narrower than said first range.

3. The rendering device according to claim 1, wherein

said second viewpoint is a virtual viewpoint previously set above said vehicle, and

said second image generating part executes viewpoint conversion processing onto the captured image received by said reception part, and generates the upper viewpoint image showing said surrounding region viewed from said virtual viewpoint.

4. The rendering device according to claim 1, wherein

said first viewpoint is a virtual viewpoint previously set in close proximity to said image capture device, and

said first image generating part executes viewpoint conversion processing onto the captured image received by said reception part, and generates the lower viewpoint image showing said surrounding region viewed from said virtual viewpoint.

5. The rendering device according to claim 1, wherein

said first viewpoint is a virtual viewpoint previously set in close proximity to an eye position of a driver of said vehicle, and

said first image generating part executes viewpoint conversion processing onto the captured image received by said reception part, and generates the lower viewpoint image showing said surrounding region viewed from said virtual viewpoint.

6. The rendering device according to claim 1, wherein said third image generating part arranges the upper viewpoint image generated by said second image generating part on a side of the drive assistant image closer to a driver'"'"'s seat in said vehicle.

7. The rendering device according to claim 1, wherein said third image generating part arranges the upper viewpoint image generated by said second image generating part on an upper side of the drive assistant image.

8. The rendering device according to claim 1, further comprising:

a derivation part for deriving, based on a rudder angle of said vehicle detected by a rudder angle sensor placed in the vehicle, an estimated path for said vehicle to take in a course of time; and

a path rendering part for rendering, on the upper viewpoint image and/or the lower viewpoint image structuring the drive assistant image generated by said third image generating part, an object indicative of the estimated path derived by said derivation part.

9. A rendering method for generating a drive assistant image for drive assistance, comprising:

a receiving step of receiving an image captured by an image capture device placed in a vehicle;

a first image generating step of generating, based on the captured image received in said receiving step, a lower viewpoint image showing a surrounding region of said vehicle viewed from a first viewpoint;

a second image generating step of generating, based on the captured image received in said receiving step, an upper viewpoint image showing said surrounding region viewed from a second viewpoint differing from said first viewpoint; and

a third image generating step of generating the drive assistant image including the lower viewpoint image generated in said first image generating step, and the upper viewpoint image generated in said second image generating step;

wherein

a first angle formed between a road surface and a direction of said first viewpoint to said surrounding region is smaller than a second angle formed between the road surface and a direction of said second viewpoint to said surrounding region.

10. The rendering method according to claim 9, wherein

in said first image generating step, the lower viewpoint image showing a first range in said surrounding region viewed from said first viewpoint is generated,

in said second image generating step, the upper viewpoint image showing a second range in said surrounding region viewed from said second viewpoint is generated, and

said second range is narrower than said first range.

11. The rendering method according to claim 9, wherein

said second viewpoint is a virtual viewpoint previously set above said vehicle, and

in said second image generating step, viewpoint conversion processing is executed to the captured image received in said receiving step, and the upper viewpoint image showing said surrounding region viewed from said virtual viewpoint is generated.

12. The rendering method according to claim 9, wherein

said first viewpoint is a virtual viewpoint previously set in close proximity to said image capture device, and

in said first image generating step, viewpoint conversion processing is executed to the captured image received in said reception step, and the lower viewpoint image showing said surrounding region viewed from said virtual viewpoint is generated.

13. The rendering method according to claim 9, wherein

said first viewpoint is a virtual viewpoint previously set in close proximity to an eye position of a driver of said vehicle, and

in said first image generating step, viewpoint conversion processing is executed to the captured image received in said receiving step, and the lower viewpoint image showing said surrounding region viewed from said virtual viewpoint is generated.

14. A recording medium with a program recorded for generating a drive assistant image for drive assistance, said program comprising:

a receiving step of receiving an image captured by an image capture device placed in a vehicle;

a first image generating step of generating, based on the captured image received in said receiving step, a lower viewpoint image showing a surrounding region of said vehicle viewed from a first viewpoint;

a second image generating step of generating, based on the captured image received in said receiving step, an upper viewpoint image showing said surrounding region viewed from a second viewpoint differing from said first viewpoint; and

a third image generating step of generating the drive assistant image including the lower viewpoint image generated in said first image generating step, and the upper viewpoint image generated in said second image generating step;

wherein

a first angle formed between a road surf ace and a direction of said first viewpoint to said surrounding region is smaller than a second angle formed between the road surface and a direction ht of said second viewpoint to said surrounding region.

15. The recording medium with the program recorded according to claim 14, wherein

in said first image generating step, the lower viewpoint image showing a first range in said surrounding region viewed from said first viewpoint is generated,

in said second image generating step, the upper viewpoint image showing a second range in said surrounding region viewed from said second viewpoint is generated, and

said second range is narrower than said first range.

16. The recording medium with the program recorded according to claim 14, wherein

said second viewpoint is a virtual viewpoint previously set above said vehicle, and

in said second image generating step, viewpoint conversion processing is executed to the captured image received in said receiving step, and the upper viewpoint image showing said surrounding region viewed from said virtual viewpoint is generated.

17. The recording medium with the program recorded according to claim 14, wherein

said first viewpoint is a virtual viewpoint previously set in close proximity to said image capture device, and

in said first image generating step, viewpoint conversion processing is executed to the captured image received in said reception step, and the lower viewpoint image showing said surrounding region viewed from said virtual viewpoint is generated.

18. The recording medium with the program recorded according to claim 14, wherein

said first viewpoint is a virtual viewpoint previously set in close proximity to an eye position of a driver of said vehicle, and

in said first image generating step, viewpoint conversion processing is executed to the captured image received in said receiving step, and the lower viewpoint image showing said surrounding region viewed from said virtual viewpoint is generated.

19. A program for generating a drive assistant image for drive assistance, comprising:

a receiving step of receiving an image captured by an image capture device placed in a vehicle;

a first image generating step of generating, based on the captured image received in said receiving step, a lower viewpoint image showing a surrounding region of said vehicle viewed from a first viewpoint;

a second image generating step of generating, based on the captured image received in said receiving step, an upper viewpoint image showing said surrounding region viewed from a second viewpoint differing from said first viewpoint; and

a third image generating step of generating the drive assistant image including the lower viewpoint image generated in said first image generating step, and the upper viewpoint image generated in said second image generating step;

wherein

a first angle formed between a road surface and a direction of said first viewpoint to said surrounding region is smaller than a second angle formed between the road surface and a direction of said second viewpoint to said surrounding region.

20. The program according to claim 19, wherein

in said first image generating step, the lower viewpoint image showing a first range in said surrounding region viewed from said first viewpoint is generated,

in said second image generating step, the upper viewpoint image showing a second range in said surrounding region viewed from said second viewpoint is generated, and

said second range is narrower than said first range.

21. The program according to claim 19, wherein

said second viewpoint is a virtual viewpoint previously set above said vehicle, and

in said second image generating step, viewpoint conversion processing is executed to the captured image received in said receiving step, and the upper viewpoint image showing said surrounding region viewed from said virtual viewpoint is generated.

22. The program according to claim 19, wherein

said first viewpoint is a virtual viewpoint previously set in close proximity to said image capture device, and

in said first image generating step, viewpoint conversion processing is executed to the captured image received in said reception step, and the lower viewpoint image showing said surrounding region viewed from said virtual viewpoint is generated.

23. The program according to claim 19, wherein

said first viewpoint is a virtual viewpoint previously set in close proximity to an eye position of a driver of said vehicle, and

in said first image generating step, viewpoint conversion processing is executed to the captured image received in said receiving step, and the lower viewpoint image showing said surrounding region viewed from said virtual viewpoint is generated.