Image generating apparatus and image generating method

• US 8,259,173 B2
• Filed: 09/26/2007
• Issued: 09/04/2012
• Est. Priority Date: 09/26/2006
• Status: Active Grant

• Current Assignee: Clarion Co., Ltd. (Faurecia SA)
• Original Assignee: Clarion Co., Ltd. (Faurecia SA)
• Inventors: Iwano, Hirotaka
• Primary Examiner(s): Etienne, Ario
• Assistant Examiner(s): Kim, Hee Soo
• Application Number:
  US11/902,952
• Publication Number:
  US 20080136912A1
• Time in Patent Office: 1,805 Days
• Field of Search: 348/148, 348/143
• US Class Current: 348/148
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/605 the adjustment being automatic
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8066 for monitoring rearward traffic
• CPC Class Codes: G06T 1/0007 Image acquisition

Abstract

An image generating apparatus, which generates monitor image data from camera image data through a viewpoint transformation, includes a rear camera, an image processing controller, and a monitor. The image processing controller sets a virtual stereoscopic imaging plane as a virtual imaging plane of a virtual camera set at a position higher than the rear camera, transfers each picture element by a coordinate transformation between a position of each picture element of single camera image data captured by the rear camera and a position of each picture element on the virtual stereoscopic imaging plane of the virtual camera through a virtual projection plane set on ground, and generates corresponding monitor image data.

Claims

1. An image generating apparatus, comprising:

a real camera which captures camera image data projected onto a single real imaging plane and ranging from a close view to a distant view;

an image processing controller

which sets a virtual camera at a position higher than the real camera,

which sets a virtual projection plane from the virtual camera on ground, which sets a virtual stereoscopic imaging plane, as a virtual imaging plane of the virtual camera, including a close view imaging plane parallel to the ground and a distant view imaging plane connected to a side of the close view imaging plane near from the real camera and set to have a predetermined downward tilt angle relative to the ground,

which performs a coordinate transformation between a position of each picture element of the single camera image data captured by the real camera and a position of each picture element on the virtual stereoscopic imaging plane of the virtual camera through the virtual projection plane,

which transfers the each picture element of the camera image data of the close view onto the close view imaging plane and transfers the each picture element of the camera image data of the distant view connected to the each picture element of the camera image data of the close view onto the distant view imaging plane connected to the close view imaging plane and tilted relative to the ground according to the coordinate transformation, and

which generates monitor image data for obtaining a monitor image, the monitor image being as an image to be projected on the virtual stereoscopic imaging plane when the virtual projection plane is viewed from the virtual camera; and

a monitor which displays a synthesis image in which a downward image of the close view and a perspective image of an intermediate view to the distant view projected by the monitor image data generated by the image processing controller are synthesized.

2. The image generating apparatus according to claim 1, wherein the image processing controller sets the distant view imaging plane of the virtual stereoscopic imaging plane of the virtual camera to be in parallel with the real imaging plane of the real camera.

3. The image generating apparatus according to claim 1,

wherein a mapping table is generated by performing a coordinate transformation of a position of each picture element in an order of:

designating one position of the picture element on the virtual stereoscopic imaging plane of the virtual camera;

determining a first corresponding position on the virtual projection plane, corresponding to the designated one position of the picture element; and

determining a second corresponding position on the imaging plane of the real camera, corresponding to the first corresponding position, and

wherein the image processing controller performs a coordinate transformation, by using the mapping table, between the position of the each picture element of the single camera image data captured by the real camera and the position of the each picture element on the virtual stereoscopic imaging plane of the virtual camera.

4. The image generating apparatus according to claim 1,

wherein a transformation relationship characteristic indicating a relationship between a distance of a picture element from an optical axis by a distorted image of the camera image data and a distance of a picture element from an optical axis by a non-distorted image of the camera image data is determined in advance, and

wherein the image processing controller corrects a distortion of a coordinate system of the position of the each picture element of the single camera image data captured by the real camera, by using the distance of the each picture element from the optical axis and the transformation relationship characteristic, to transform it into a coordinate system of the virtual stereoscopic imaging plane.

5. The image generating apparatus according to claim 1, further comprising a virtual camera setting operation member connected with the image processing controller and which sets the virtual camera on an arbitrary spatial position in accordance with an external operation.

6. The image generating apparatus according to claim 1, further comprising a virtual stereoscopic imaging plane setting operation member connected with the image processing controller and which arbitrarily sets the downward tilt angle of the distant view imaging plane to the close view imaging plane in accordance with an external operation.

7. The image generating apparatus according to claim 1, wherein the real camera includes a rear camera attached to a rear part of a vehicle and which images a view behind the vehicle.

8. An image generating method, comprising:

capturing camera image data projected onto a single real imaging plane of a real camera and ranging from a close view to a distant view;

setting a virtual camera at a position higher than the real camera;

setting a virtual projection plane of the virtual camera on ground;

setting a virtual stereoscopic imaging plane, as a virtual imaging plane of the virtual camera, including a close view imaging plane parallel to the ground and a distant view imaging plane connected to a side of the close view imaging plane near from the real camera and set to have a predetermined downward tilt angle relative to the ground;

performing a coordinate transformation between a position of each picture element of the single camera image data captured by the real camera and a position of each picture element on the virtual stereoscopic imaging plane of the virtual camera through the virtual projection plane;

generating monitor image data for obtaining a monitor image, the monitor image being as an image to be projected on the virtual stereoscopic imaging plane when the virtual projection plane is viewed from the virtual camera, by transferring the each picture element of the camera image data of the close view onto the close view imaging plane and transferring the each picture element of the camera image data of the distant view connected to the each picture element of the camera image data of the close view onto the distant view imaging plane connected to the close view imaging plane and tilted relative to the ground according to the coordinate transformation; and

displaying on a monitor a synthesis image in which a downward image of the close view and a perspective image of an intermediate view to the distant view projected by the generated monitor image data are synthesized.

9. The image generating method according to claim 8, wherein the distant view imaging plane of the virtual stereoscopic imaging plane of the virtual camera is set to be in parallel with the real imaging plane of the real camera.

10. The image generating method according to claim 8, wherein

a mapping table is generated by performing a coordinate transformation of a position of each picture element in an order of:

designating one position of the picture element on the virtual stereoscopic imaging plane of the virtual camera;

determining a first corresponding position on the virtual projection plane, corresponding to the designated one position of the picture element; and

determining a second corresponding position on the imaging plane of the real camera, corresponding to the first corresponding position, and

wherein the method performs a coordinate transformation, by using the mapping table, between the position of the each picture element of the single camera image data captured by the real camera and the position of the each picture element on the virtual stereoscopic imaging plane of the virtual camera.

11. The image generating method according to claim 8,

wherein a transformation relationship characteristic indicating a relationship between a distance of a picture element from an optical axis by a distorted image of the camera image data and a distance of a picture element from an optical axis by a non-distorted image of the camera image data is determined in advance, and

wherein the method corrects a distortion of a coordinate system of the position of the each picture element of the single camera image data captured by the real camera, by using the distance of the each picture element from the optical axis and the transformation relationship characteristic, to transform it into a coordinate system of the virtual stereoscopic imaging plane.

12. The image generating method according to claim 8, further comprising setting the virtual camera on an arbitrary spatial position in accordance with an external operation to a virtual camera setting operation member.

13. The image generating method according to claim 8, further comprising

setting the downward tilt angle of the distant view imaging plane to the close view imaging plane in accordance with an external operation to a virtual stereoscopic imaging plane setting operation member.

14. The image generating method according to claim 8, wherein the camera image data is captured by a rear camera attached to a rear part of a vehicle and which images a view behind the vehicle.