VEHICLE IMAGE PROCESSING APPARATUS AND VEHICLE IMAGE PROCESSING METHOD

• US 20150232031A1
• Filed: 05/04/2015
• Published: 08/20/2015
• Est. Priority Date: 05/29/2008
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Shimizu, Seiya, Yamada, Hiroshi, KITAURA, Asako
• Granted Patent: US 9,403,483 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60K 2370/152 Displays
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/08 Projecting images onto non-planar surfaces, e.g. geodetic screens
• CPC Class Codes: G06T 3/10 Selection of transformation methods according to the characteristics of the input images
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A vehicle image processing apparatus includes a group of cameras, a drawing unit that converts a captured image into an image viewed along a line of sight running from a predetermined position in a predetermined direction, a viewing-line-of-sight changing unit that detects whether a first line of sight is switched to a second line of sight, and a viewing-line-of-sight generation/updating unit that acquires parameters concerning the first and second lines of sight after detecting the switching and generates a parameter which is gradually changed from the parameter of the first line of sight to the parameter of the second line of sight. Moreover, the drawing unit generates, on the basis of the gradually changed parameter, an image which is gradually changed from an image viewed along the first line of sight to an image viewed along the second line of sight.

Claims

1. A vehicle image processing apparatus comprising:

a distortion correction unit that acquires a captured image which is an image that has captured an area around a vehicle with use of at least one camera;

a drawing unit that uses at least one predetermined projection shape to convert the captured image into an image viewed along a virtual line of sight that is a line of sight running from a predetermined position in a predetermined direction;

a viewing-line-of-sight changing unit that detects whether a virtual line of sight is switched from a position of a first virtual line of sight to a position of a second virtual line of sight which is a different virtual line of sight, the position of the first virtual line of sight associated with a parameter value of at least one selected from a predetermined position and a direction of the virtual line of sight and the position of the second virtual line of sight being associated with a parameter value different from the parameter value associated with the position of the first virtual line of sight;

a viewing-line-of-sight generation/updating unit that acquires at least one type of parameter value associated with a virtual line of sight for each position of the first virtual line of sight and the second virtual line of sight after detecting that the virtual line of sight is switched from the position of the first virtual line of sight to the position of the second virtual line of sight, and generates a parameter value of a position of a virtual line of sight that is gradually changed along a predetermined path connecting the position of the first virtual line of sight and the position of the second virtual line of sight; and

a projection image conversion unit that converts the captured image to an image viewed along the first virtual line of sight using a projection map which includes a relationship between a position of a pixel of the captured image and a position of a pixel of an image viewed along the first virtual line of sight, and generates the converted image to be viewed from the second virtual line of sight using a predetermined projection shape including display of a movement path of the predetermined projection shape from the first virtual line of sight to the second virtual line of sight,

wherein the drawing unit illustrates the movement path of the predetermined projection shape in the converted image based on the generated parameter value[ so that the converted image is gradually changed from a first image viewed along the position of the first virtual line of sight to a second image viewed along the position of the second virtual line of sight, and

wherein the viewing-line-of-sight changing unit detects that the position of the first virtual line of sight is switched to the position of the second virtual line of sight when a switching occurs of a pre-registered virtual line of sight or when a difference between unchanged and changed virtual lines of sight is greater than or equal to a specified value.

2. The vehicle image processing apparatus according to claim 1, wherein the at least one type of parameter value acquired includes a visual point position, viewing direction, angle of field and focal distance for the first virtual line of sight and second virtual lines of sight, as the parameter values, and

wherein the viewing-line-of-sight generation/updating unit uses a predetermined interpolation method to generate the generated parameter value.

3. The vehicle image processing apparatus according to claim 1, comprising:

a related object determination unit that determines a related object to be displayed along with at least the first image, second image or changed image from among one or more related objects containing, as data, an image or shape, or both the image and shape,

wherein the related object determination unit selects a type of data, image or shape, used for the drawing of the related object determined by the related object determination unit, and

wherein the drawing unit disposes an image of the related object at least on the first image, second image or changed image based on the type of data selected by the related object determination unit, and

wherein the drawing unit disposes a shape of the related object inside a model space of the predetermined projection shape based on the type of data selected by the related object determination unit.

4. The vehicle image processing apparatus according to claim 3,

wherein the related object determination unit prioritizes and selects an image as the type of data when at least the first image, second image or changed image is an image converted by the projection image conversion unit.

5. The vehicle image processing apparatus according to claim 3, wherein the related object determination unit prioritizes and selects a shape as the type of data when at least the first image, second image or changed image is an image converted by the projection image conversion unit.

6. The vehicle image processing apparatus according to claim 3, wherein the related object is an object representing the vehicle, the camera, a blind-spot area of the camera, an obstacle around the vehicle, a position and direction of a destination of the vehicle, a traffic light around the vehicle, a sign and traffic sign, an attention area around the vehicle, feature and map information, or a travelling direction of the vehicle.

7. The vehicle image processing apparatus according to claim 3, comprising

a driving information acquisition unit and a road information acquisition unit that acquires vehicle information which is information about the vehicle,

wherein the related object determination unit determines the related object based on the vehicle information, and

wherein the projection image conversion unit disposes the related object based on the vehicle information.

8. The vehicle image processing apparatus according to claim 7, wherein the vehicle information contains at least information about driving of the vehicle.

9. The vehicle image processing apparatus according to claim 7, wherein the vehicle information contains at least information about a road around the vehicle.

10. The vehicle image processing apparatus according to claim 7, wherein the vehicle information contains at least information about an obstacle on a road around the vehicle.

11. The vehicle image processing apparatus according to claim 1, wherein either the position of the first virtual line of sight or the second virtual line of sight respectively is a virtual line of sight associated with the predetermined position being a position of a visual point of a driver who drives the vehicle.

12. The vehicle image processing apparatus according to claim 1, wherein a line of sight that is close to a line of sight of a driver, corresponds with one of the position of the first line of sight and the position of the second line of sight including an associated parameter or for the generated parameter value.

13. A vehicle image processing method comprising:

acquiring a captured image which is an image that has captured an area around a vehicle with use of at least one camera;

converting the captured image to an image viewed along a first virtual line of sight using a projection map which includes a relationship between a position of a pixel of the captured image and a position of a pixel of an image viewed along the first virtual line of sight, and generating the converted image to be viewed from a second virtual line of sight, different from the first virtual line of sight, using a predetermined projection shape including display of a movement path of the predetermined projection shape from the first virtual line of sight to the second virtual line of sight;

detecting whether a virtual line of sight is switched from a position of the first virtual line of sight to a position of the second virtual line of sight, the position of the first virtual line of sight associated with a parameter value of at least one selected from a predetermined position and a direction of the first virtual line of sight and the position of the second virtual line of sight being associated with a parameter value different from the parameter value associated with the position of the first virtual line of sight;

acquiring at least one type of parameter value associated with a virtual line of sight for each position of the first virtual line of sight and the second virtual line of sight, respectively, after detecting that the virtual line of sight is switched from the position of the first virtual line of sight to the position of second virtual line of sight, and generating a parameter value of a position of a virtual line of sight that is gradually changed along a predetermined path connecting the position of the first virtual line of sight and the position of the second virtual line of sight; and

illustrating the movement path of the predetermined projection shape in the converted image based on the generated parameter value so that the converted image is gradually changed from a first image viewed along the position of the first virtual line of sight to a second image viewed along the position of the second virtual line of sight, and

wherein the detecting detects that the position of the first virtual line of sight is switched to the position of the second virtual line of sight when a switching occurs of a pre-registered virtual line of sight or when a difference between unchanged and changed virtual lines of sight is greater than or equal to a specified value.

14. A vehicle image processing apparatus comprising:

a processor to execute;

detecting whether a virtual line of sight is switched from a position of a first virtual line of sight having a parameter value to a position of a second virtual line of sight having a parameter value different from the parameter value of the position of the first virtual line of sight,

converting a captured image to an image viewed along the first virtual line of sight using a projection map which includes a relationship between a position of a pixel of the captured image and a position of a pixel of an image viewed along the first virtual line of sight, and generating the converted image to be viewed from the second virtual line of sight using a predetermined projection shape including display of a movement path of the predetermined projection shape from the first virtual line of sight to the second virtual line of sight,

generating a parameter value of a position of a virtual line of sight that is gradually changed along a predetermined path connecting the position of the first virtual line of sight and the position of the second virtual line of sight,

illustrating the movement path of the predetermine projection shape in the converted image based on the generated parameter value, and

wherein the detecting indicates the virtual line of sight is switched when a pre-registered virtual line of sight is switched or when a difference between virtual lines of sight is greater than or equal to a specified value.