Image generation for assistance of drivers of vehicles

• US 6,999,602 B2
• Filed: 06/28/2001
• Issued: 02/14/2006
• Est. Priority Date: 06/30/2000
• Status: Expired due to Fees

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Yoshida, Takashi, Iisaka, Atsushi, Ishida, Akira, Yasui, Nobuhiko
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): Edwards, Patrick L.
• Application Number:
  US09/892,741
• Publication Number:
  US 20020001399A1
• Time in Patent Office: 1,692 Days
• Field of Search: 382103-104, 382/284, 340901-996, 3404255-490, 340/146.2, 340/158, 340/853.6, 340/572.4, 701/28, 701/41, 701/42, 701/44, 348/119, 348/148, 348/149, 116/31, 116/36
• US Class Current: 382/104
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/8086 for vehicle path indication
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation

Abstract

In a rendering device Urnd1, a CPU 7 receives a captured image each from two image capture devices 1 and 2. The CPU 7 then receives a steering angle of a steering wheel of a vehicle from a steering angle sensor 3. Based on the steering angle, the CPU 7 derives an estimated trajectory for the vehicle to take in the course of time. Then, with reference to the estimated trajectory, the CPU 7 selects several pixels each from the captured images, and then generates a drive assistant image based on those pixels. In this manner, the drive assistant image generated by the rendering device Urnd1 hardly causes the driver to feel strange.

Claims

1. A rendering device for generating a drive assistant image of an area around a vehicle for drive assistance, said device comprising:

an image receiving part operable to receive images captured by a plurality of image capture devices, which are fixed in the vehicle respectively, operable to capture images of the area around the vehicle, the captured images include at least one overlapped region;

a steering angle receiving part operable to receive a steering angle of the vehicle detected by a steering angle sensor fixed in the vehicle;

a trajectory deriving part operable to derive an estimated trajectory for the vehicle to take based on the steering angle received by said steering angle receiving part; and

an image processing part operable to perform pixel selection from the captured images received by said image receiving part according to the steering angle received by said steering angle receiving part, and based on a result of the pixel selection, to generate the drive assistant image, wherein

if the estimated trajectory derived by said trajectory deriving part is designated over an overlapped region, said image processing part is operable to select pixels from a captured image received by said image receiving part representing one side of the overlapped region with respect to an imaginary line, and to select pixels from another captured image representing the other side of the overlapped region with respect to the imaginary line, and

the imaginary line is one selected from a group including the estimated trajectory, a line displaced by a predetermined amount parallel to the estimated trajectory, and a chord of the estimated trajectory.

2. The rendering device according to claim 1, further comprising a table storing part operable to store a mapping table showing a correspondence between the drive assistant image and the captured images on a pixel basis, wherein

in the mapping table, a pixel belonging to the overlapped region in the drive assistant image corresponds to a plurality of pixels in the captured images according to the steering angle received by said steering angle receiving part, and

according to the mapping table stored in said table storing part, said image processing part selects the pixels from each of the captured images received by said image receiving part.

3. The rendering device according to claim 1, further comprising:

a trajectory rendering part operable to render the estimated trajectory derived by said trajectory deriving part on the drive assistant image generated by said image processing part.

4. The rendering device according to claim 1, further comprising:

an image storing part operable to store a vehicle image representing the vehicle; and

a vehicle rendering part operable to render the vehicle image stored in said image storing part on the drive assistant image generated by said image processing part.

5. The rendering device according to claim 1, wherein said image processing part generates the drive assistant image showing the area around the vehicle viewed from a predetermined virtual camera.

6. A rendering method for generating a drive assistant image of an area around a vehicle for drive assistance, said method comprising:

receiving images captured by a plurality of image capture devices, which are fixed in the vehicle respectively, operable to capture images of the area around the vehicle, the captured images include at least one overlapped region;

receiving a steering angle of the vehicle detected by a steering angle sensor fixed in the vehicle;

deriving an estimated trajectory for the vehicle to take based on the received steering angle; and

performing pixel selection from the received images according to the received steering angle, and based on a result of the pixel selection, generating the drive assistant image, wherein

if the estimated trajectory is designated an overlapped region, in said pixel selection, pixels are selected from a captured image representing one side of the overlapped region with respect to an imaginary line, and pixels are selected from another captured image representing the other side of the overlapped region with respect to the imaginary line, and

the imaginary line is one selected from a group including the estimated trajectory, a line displaced by a predetermined amount parallel to the estimated trajectory, and a chord of the estimated trajectory.

7. The rendering method according to claim 6, further comprising storing a mapping table showing a correspondence between the drive assistant image and the captured images on a pixel basis, wherein

in the mapping table, a pixel belonging to the overlapped region in the drive assistant image corresponds to a plurality of pixels in the captured images according to the received steering angle, and

according to the stored mapping table, in said pixel selection, the pixels are selected from each of the received images.

8. A recording medium having a program recorded thereon for generating a drive assistant image of an area around a vehicle for drive assistance, said program including system readable instructions capable of instructing a system to perform the method comprising:

receiving images captured by a plurality of image capture devices, which are fixed in the vehicle respectively, operable to capture images of the area around the vehicle, the captured images include at least one overlapped region;

receiving a steering angle of the vehicle detected by a steering angle sensor fixed in the vehicle;

deriving an estimated trajectory for the vehicle to take based on the received steering angle; and

performing pixel selection from the received images according to the steering angle, and based on a result of the pixel selection, generating the drive assistant image, wherein

if the estimated trajectory is designated an overlapped region, in said pixel selection, pixels are selected from a captured image representing one side of the overlapped region with respect to an imaginary line, and pixels are selected from another captured image representing the other side of the overlapped region with respect to the imaginary line, and

the imaginary line is one selected from a group including the estimated trajectory, a line displaced by a predetermined amount parallel to the estimated trajectory, and a chord of the estimated trajectory.

9. The recording medium having the program recorded thereon according to claim 8, further including system readable instructions capable of instructing a system to additionally store a mapping table showing a correspondence between the drive assistant image and the captured images on a pixel basis, wherein

in the mapping table, a pixel belonging to the overlapped region in the drive assistant image corresponds to a plurality of pixels in the captured images according to the received steering angle, and

according to the stored mapping table, in said pixel selection, the pixels are selected from each of the received captured images.