VEHICLE IMAGE PROCESSING APPARATUS AND VEHICLE IMAGE PROCESSING METHOD

• US 20110032357A1
• Filed: 10/19/2010
• Published: 02/10/2011
• Est. Priority Date: 05/29/2008
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Yamada, Hiroshi, Shimizu, Seiya, Kitaura, Asako
• Granted Patent: US 9,475,430 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: B60K 2370/152 Displays
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/08 Projecting images onto non-planar surfaces, e.g. geodetic screens
• CPC Class Codes: G06T 3/10 Selection of transformation methods according to the characteristics of the input images
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/006 Geometric correction correction of chromatic aberrations adapted to be used in an image pickup device containing an electronic image sensor H04N23/10; detecting, correcting, reducing or removing artefacts resulting only from the lens unit, e.g. flare, shading, vignetting or "cos4" H04N25/61
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A vehicle image processing apparatus includes: a group of cameras; a drawing unit that converts a captured image into an image viewed along a line of sight running from a predetermined position in a predetermined direction; a viewing-line-of-sight changing unit that detects whether a first line of sight is switched to a second line of sight; and a viewing-line-of-sight generation/updating unit that acquires parameters concerning the first and second lines of sight after detecting the switching and generates a parameter which is gradually changed from the parameter of the first line of sight to the parameter of the second line of sight. Moreover, the drawing unit generates, on the basis of the gradually changed parameter, an image which is gradually changed from an image viewed along the first line of sight to an image viewed along the second line of sight.

Claims

1. A vehicle image processing apparatus comprising:

a distortion correction unit that acquires a captured image which is an image that has captured an area around a vehicle with the use of at least one camera;

a drawing unit that uses at least one predetermined projection shape to convert the captured image into an image viewed along a virtual line of sight that is a line of sight running from a predetermined position in a predetermined direction;

a viewing-line-of-sight changing unit that detects whether a first virtual line of sight which is a predetermined virtual line of sight is switched to a second virtual line of sight which is a different virtual line of sight from the first virtual line of sight; and

a viewing-line-of-sight generation/updating unit that acquires at least one type of parameter value concerning the virtual lines of sight each for the first and second virtual lines of sight after detecting that the first virtual line of sight is switched to the second virtual line of sight, and generating a parameter value that is gradually changed from the parameter value of the first virtual line of sight to the parameter value of the second virtual line of sight;

wherein

the drawing unit generates, on the basis of the gradually changed parameter value, at least one changed image which is an image that is gradually changed from a first image which is an image viewed along the first virtual line of sight to a second image which is an image viewed along the second virtual line of sight, wherein

the viewing-line-of-sight changing unit detects that the first virtual line of sight is switched to the second virtual line of sight if the switching is the switching of a virtual line of sight which is registered in advance or if the difference between the unchanged and changed virtual lines of sight is greater than or equal to a specified value.

2. The vehicle image processing apparatus according to claim 1, wherein

the viewing-line-of-sight generation/updating unit acquires one or more of the factors, a visual point position, viewing direction, angle of field and focal distance for the first and second virtual lines of sight, as the parameter values, and uses a predetermined interpolation method to generate the gradually changed parameter value.

3. The vehicle image processing apparatus according to claim 2, wherein

the viewing-line-of-sight generation/updating unit generates the gradually changed parameter value on the basis of a reference point where the vehicle is.

4. The vehicle image processing apparatus according to claim 2, wherein

the viewing-line-of-sight generation/updating unit generates the gradually changed parameter value on the basis of a reference point which is calculated with the use of the line-of-sight directions of the first and second virtual lines of sight.

5. The vehicle image processing apparatus according to claim 1, wherein

the drawing unit converts the captured image to an image viewed along the virtual line of sight with the use of a projection map which at least holds the relationship between the position of a pixel of the captured image and the position of a pixel of an image viewed along the virtual line of sight, wherein

at least either the first or second image is an image converted by the drawing unit, wherein

the drawing unit generates the changed image using the drawing unit.

6. The vehicle image processing apparatus according to claim 1, further comprising:

a related object determination unit that determines a related object to be displayed along with at least the first image, second image or changed image from among one or more related objects containing, as data, an image or shape, or both the image and shape;

wherein

the related object determination unit selects the type of data, image or shape, used for the drawing of the related object determined by the related object determination unit, wherein

the drawing unit disposes an image of the related object at least on the first image, second image or changed image on the basis of the type of data selected by the related object determination unit, wherein

the drawing unit disposes a shape of the related object inside a model space of the projection shape on the basis of the type of data selected by the related object determination unit.

7. The vehicle image processing apparatus according to claim 6, wherein

the drawing unit converts the captured image to an image viewed along the virtual line of sight with the use of a projection map which at least holds the relationship between the position of a pixel of the captured image and the position of a pixel of an image viewed along the virtual line of sight, wherein

the related object determination unit prioritizes and selects an image as the type of data when at least the first image, second image or changed image is an image converted by the drawing unit.

8. The vehicle image processing apparatus according to claim 6, wherein

the related object determination unit prioritizes and selects shape as the type of data when at least the first image, second image or changed image is an image converted by the drawing unit.

9. The vehicle image processing apparatus according to claim 6, wherein

the related object is an object representing the vehicle, the camera, a blind-spot area of the camera, an obstacle around the vehicle, the position and direction of a destination of the vehicle, a traffic light around the vehicle, a sign and traffic sign, an attention area around the vehicle, feature and map information, or the travelling direction of the vehicle.

10. The vehicle image processing apparatus according to claim 6, further comprising

a driving information acquisition unit and a road information acquisition unit that acquires vehicle information which is information about the vehicle, wherein

the related object determination unit determines the related object on the basis of the vehicle information, wherein

the drawing unit dispose the related object on the basis of the vehicle information.

11. The vehicle image processing apparatus according to claim 10, wherein

the vehicle information contains at least information about the driving of the vehicle.

12. The vehicle image processing apparatus according to claim 10, wherein

the vehicle information contains at least information about a road around the vehicle.

13. The vehicle image processing apparatus according to claim 10, wherein

the vehicle information contains at least information about an obstacle on a road around the vehicle.

14. The vehicle image processing apparatus according to claim 1, wherein

either the first or second virtual line of sight is a virtual line of sight the predetermined position of which is the position of a visual point of a driver who drives the vehicle.

15. The vehicle image processing apparatus according to claim 1, wherein

a line of sight that is close to the line of sight of a driver, as well as a parameter of the line of sight, is used for the first or second virtual line of sight, or for the gradually changed parameter value.

16. A vehicle image processing method comprising:

acquiring a captured image which is an image that has captured an area around a vehicle with the use of at least one camera;

using at least one predetermined projection shape to convert the captured image into an image viewed along a virtual line of sight that is a line of sight running from a predetermined position in a predetermined direction;

detecting whether a first virtual line of sight which is a predetermined virtual line of sight is switched to a second virtual line of sight which is a different virtual line of sight from the first virtual line of sight;

acquiring at least one type of parameter value concerning the virtual lines of sight each for the first and second virtual lines of sight after detecting that the first virtual line of sight is switched to the second virtual line of sight, and generating a parameter value that is gradually changed from the parameter value of the first virtual line of sight to the parameter value of the second virtual line of sight; and

generating, on the basis of the gradually changed parameter value, at least one changed image which is an image that is gradually changed from a first image which is an image viewed along the first virtual line of sight to a second image which is an image viewed along the second virtual line of sight, wherein

the detecting is to detect that the first virtual line of sight is switched to the second virtual line of sight if the switching is the switching of a virtual line of sight which is registered in advance or if the difference between the unchanged and changed virtual lines of sight is greater than or equal to a specified value.