Image generating apparatus and image generating method

• US 8,368,687 B2
• Filed: 09/26/2007
• Issued: 02/05/2013
• Est. Priority Date: 09/26/2006
• Status: Active Grant

• Current Assignee: Clarion Co., Ltd. (Faurecia SA)
• Original Assignee: Clarion Co., Ltd. (Faurecia SA)
• Inventors: Iwano, Hirotaka
• Primary Examiner(s): Wu, Xiao M.
• Assistant Examiner(s): Hoang, Phi
• Application Number:
  US11/902,951
• Publication Number:
  US 20080129756A1
• Time in Patent Office: 1,959 Days
• Field of Search: 382/104, 348/148, 348/118
• US Class Current: 345/419
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 1/31 providing stereoscopic vision
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06T 3/053 Detail-in-context presentations fisheye or wide-angle transformations G06T3/047

Abstract

An image generating apparatus, which generates monitor image data from camera image data through a viewpoint transformation, includes a rear camera 1, an image processing controller 2, and a monitor 3. The image processing controller 2 sets a first virtual camera by viewpoint shift to have a downward viewpoint from a higher position than that of the rear camera 1, and sets a second virtual camera by rotating a viewpoint thereof upward or downward from the viewpoint of the rear camera 1, and generates the monitor image data of a close view viewed from the first virtual camera, and of an intermediate-to-distant view viewed from the second virtual camera, from the single camera image data captured by the rear camera 1. The monitor 3 displays a synthesis image in which a downward view of the close view and a perspective image of the intermediate-to-distant view are synthesized.

Claims

1. An image generating apparatus, comprising:

a single real camera which captures camera image data projected onto a single real imaging plane and ranging from a close view to a distant view; and

an image processing controller

which sets a single virtual camera at a position higher than the single real camera,

which sets, on an optical axis of the single real camera, a virtual stereoscopic projection plane, as a virtual projection plane from the single virtual camera, formed in a stereoscopic screen including a close view screen set on ground and a distant view screen being connected to a side of the close view screen far from the single real camera and set to have a predetermined upward tilt angle, an optical axis of the single virtual camera being disposed to be directed to the virtual stereoscopic projection plane,

which sets a virtual stereoscopic imaging plane, as a virtual imaging plane of the single virtual camera, including a close view imaging plane parallel to the close view screen and a distant view imaging plane connected to a side of the close view imaging plane near from the single real camera and set to have a predetermined downward tilt angle,

which performs a coordinate transformation between a position of each picture element of the single camera image data captured by the single real camera and a position of each picture element on the virtual imaging plane of the single virtual camera through the virtual stereoscopic projection plane,

which transfers the each picture element of the camera image data onto the virtual stereoscopic imaging plane of the single virtual camera according to the coordinate transformation, and

which generates, from the single camera image data captured by the single real camera, monitor image data for obtaining a monitor image, the monitor image being as an image to be projected on the virtual stereoscopic imaging plane when the virtual stereoscopic projection plane is viewed from the single virtual camera; and

a monitor which displays a synthesis image in which a downward image of the close view and a perspective image of an intermediate view to the distant view projected by the monitor image data generated from the single camera image data captured by the single real camera by the image processing controller are seamlessly connected to be synthesized to have continuity in the synthesis image.

2. An image generating method, comprising:

capturing camera image data projected onto a single real imaging plane of a single real camera and ranging from a close view to a distant view;

setting a single virtual camera at a position higher than the single real camera;

setting, on an optical axis of the single real camera, a virtual stereoscopic projection plane, as a virtual projection plane, formed in a stereoscopic screen including a close view screen set on ground and a distant view screen being connected to a side of the close view screen far from the single real camera and set to have a predetermined upward tilt angle, an optical axis of the single virtual camera being disposed to be directed to the virtual stereoscopic projection plane;

performing a coordinate transformation between a position of each picture element of the single camera image data captured by the single real camera and a position of each picture element on a virtual imaging plane of the single virtual camera through the virtual stereoscopic projection plane;

generating, from the single camera image data captured by the single real camera, monitor image data for obtaining a monitor image, the monitor image being as an image to be projected on the virtual imaging plane when the virtual stereoscopic projection plane is viewed from the single virtual camera, by transferring the each picture element of the camera image data onto the virtual imaging plane of the virtual camera according to the coordinate transformation; and

displaying on a monitor a synthesis image in which a downward image of the close view and a perspective image of an intermediate view to the distant view projected by the monitor image data generated from the single camera image data captured by the single real camera are seamlessly connected to be synthesized to have continuity in the synthesis image.

3. The image generating method according to claim 2, wherein

a mapping table is generated by performing a coordinate transformation of a position of each picture element in an order of:

designating one position of the picture element on a virtual imaging plane of the single virtual camera;

determining a first corresponding position on a virtual projection plane, corresponding to the designated one position of the picture element; and

determining a second corresponding position on the imaging plane of the single real camera, corresponding to the first corresponding position, and

wherein the method performs a coordinate transformation, by using the mapping table, between the position of the each picture element of the single camera image data captured by the single real camera and the position of the each picture element on the virtual imaging plane of the single virtual camera.

4. The image generating method according to claim 2, wherein a transformation relationship characteristic indicating a relationship between a distance of a picture element from an optical axis by a distorted image of the camera image data and a distance of a picture element from an optical axis by a non-distorted image of the camera image data is determined in advance, and

wherein the method corrects a distortion of a coordinate system of the position of the each picture element of the single camera image data captured by the single real camera, by using the distance of the each picture element from the optical axis and the transformation relationship characteristic, to transform it into a coordinate system of the virtual imaging plane.

5. The image generating method according to claim 2, further comprising

setting the single virtual camera on an arbitrary spatial position in accordance with an external operation to a virtual camera setting operation member.

6. The image generating method according to claim 2, further comprising

setting an upward tilt angle of a screen for the distant view to a screen for the close view in accordance with an external operation to a virtual stereoscopic projection plane setting operation member.

7. The image generating method according to claim 2, further comprising

setting a downward tilt angle of an imaging plane for the distant view to an imaging plane for the close view in accordance with an external operation to a virtual stereoscopic imaging plane setting operation member.

8. The image generating method according to claim 2, wherein the camera image data is captured by a rear camera attached to a rear part of a vehicle and which images a view behind the vehicle.

9. An image generating method, comprising:

capturing camera image data projected onto a single real imaging plane of a single real camera and ranging from a close view to a distant view;

setting a single virtual camera at a position higher than the single real camera;

setting, on an optical axis of the single real camera, a virtual stereoscopic projection plane, as a virtual projection plane from the virtual camera, formed in a stereoscopic screen including a close view screen set on ground and a distant view screen being connected to a side of the close view screen far from the single real camera and set to have a predetermined upward tilt angle, an optical axis of the single virtual camera being disposed to be directed to the virtual stereoscopic projection plane;

setting a virtual stereoscopic imaging plane, as a virtual imaging plane of the virtual camera, including a close view imaging plane parallel to the close view screen and a distant view imaging plane connected to a side of the close view imaging plane near from the single real camera and set to have a predetermined downward tilt angle;

performing a coordinate transformation between a position of each picture element of the single camera image data captured by the single real camera and a position of each picture element on the virtual stereoscopic imaging plane of the single virtual camera through the virtual stereoscopic projection plane;

generating, from the single camera image data captured by the single real camera, monitor image data for obtaining a monitor image, the monitor image being as an image to be projected on the virtual stereoscopic imaging plane when the virtual stereoscopic projection plane is viewed from the virtual camera, by transferring the each picture element of the camera image data onto the virtual stereoscopic imaging plane of the single virtual camera according to the coordinate transformation; and

displaying on a monitor a synthesis image in which a downward image of the close view and a perspective image of an intermediate view to the distant view projected by the monitor image data generated from the single camera image data captured by the single real camera are seamlessly connected to be synthesized to have continuity in the synthesis image.