Image generating apparatus and image generating method

• US 20080129756A1
• Filed: 09/26/2007
• Published: 06/05/2008
• Est. Priority Date: 09/26/2006
• Status: Active Grant

• Current Assignee: Clarion Co., Ltd. (Faurecia SA)
• Original Assignee: Clarion Co., Ltd. (Faurecia SA)
• Inventors: Iwano, Hirotaka
• Granted Patent: US 8,368,687 B2
• Time in Patent Office: Days
• US Class Current: 345/632
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 1/31 providing stereoscopic vision
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06T 3/053 Detail-in-context presentations fisheye or wide-angle transformations G06T3/047

Abstract

An image generating apparatus, which generates monitor image data from camera image data through a viewpoint transformation, includes a rear camera 1, an image processing controller 2, and a monitor 3. The image processing controller 2 sets a first virtual camera by viewpoint shift to have a downward viewpoint from a higher position than that of the rear camera 1, and sets a second virtual camera by rotating a viewpoint thereof upward or downward from the viewpoint of the rear camera 1, and generates the monitor image data of a close view viewed from the first virtual camera, and of an intermediate-to-distant view viewed from the second virtual camera, from the single camera image data captured by the rear camera 1. The monitor 3 displays a synthesis image in which a downward view of the close view and a perspective image of the intermediate-to-distant view are synthesized.

Claims

1. An image generating apparatus, comprising:

a real camera which captures camera image data projected onto a single real imaging plane and ranging from a close view to a distant view;

an image processing controller which sets a first virtual camera in which a viewpoint thereof is shifted so as to be located higher than a viewpoint of the real camera and to face ground, and sets a second virtual camera in which a viewpoint thereof is rotated from the viewpoint of the real camera so as to face upward or downward, and which generates, from the single camera image data captured by the real camera, monitor image data for the close view viewed from the first virtual camera and monitor image data for an intermediate view to the distant view viewed from the second virtual camera; and

a monitor which displays a synthesis image in which a downward image of the close view projected by the monitor image data for the close view generated by the image processing controller and a perspective image of the intermediate view to the distant view projected by the monitor image data for the intermediate view to the distant view generated by the image processing controller are synthesized.

2. The image generating apparatus according to claim 1,

wherein a mapping table is generated by performing a coordinate transformation of a position of each picture element in an order of:

designating one position of the picture element on a virtual imaging plane of the virtual camera;

determining a first corresponding position on a virtual projection plane, corresponding to the designated one position of the picture element; and

determining a second corresponding position on the imaging plane of the real camera, corresponding to the first corresponding position, and

wherein the image processing controller performs a coordinate transformation, by using the mapping table, between the position of the each picture element of the single camera image data captured by the real camera and the position of the each picture element on the virtual imaging plane of the virtual camera.

3. The image generating apparatus according to claim 1,

wherein a transformation relationship characteristic indicating a relationship between a distance of a picture element from an optical axis by a distorted image of the camera image data and a distance of a picture element from an optical axis by a non-distorted image of the camera image data is determined in advance, and

wherein the image processing controller corrects a distortion of a coordinate system of the position of the each picture element of the single camera image data captured by the real camera, by using the distance of the each picture element from the optical axis and the transformation relationship characteristic, to transform it into a coordinate system of the virtual imaging plane.

4. The image generating apparatus according to claim 1, wherein the image processing controller is connected with a virtual camera setting operation member which sets the virtual camera on an arbitrary spatial position in accordance with an external operation.

5. The image generating apparatus according to claim 1, wherein the image processing controller is connected with a virtual stereoscopic projection plane setting operation member which arbitrarily sets an upward tilt angle of a screen for the distant view to a screen for the close view in accordance with an external operation.

6. The image generating apparatus according to claim 1, wherein the image processing controller is connected with a virtual stereoscopic imaging plane setting operation member which arbitrarily sets a downward tilt angle of an imaging plane for the distant view to an imaging plane for the close view in accordance with an external operation.

7. The image generating apparatus according to claim 1, wherein the real camera includes a rear camera attached to a rear part of a vehicle and which images a view behind the vehicle.

8. An image generating apparatus, comprising:

a real camera which captures camera image data projected onto a single real imaging plane and ranging from a close view to a distant view;

an image processing controller

which sets a virtual camera at a position higher than the real camera,

which sets a virtual stereoscopic projection plane, as a virtual projection plane from the virtual camera, including a close view screen set on ground and a distant view screen being connected to a side of the close view screen far from the real camera and set to have a predetermined upward tilt angle,

which performs a coordinate transformation between a position of each picture element of the single camera image data captured by the real camera and a position of each picture element on a virtual imaging plane of the virtual camera through the virtual stereoscopic projection plane,

which transfers the each picture element of the camera image data onto the virtual imaging plane of the virtual camera according to the coordinate transformation, and

which generates monitor image data for obtaining a monitor image, the monitor image being as an image to be projected on the virtual imaging plane when the virtual stereoscopic projection plane is viewed from the virtual camera; and

a monitor which displays a synthesis image in which a downward image of the close view and a perspective image of an intermediate view to the distant view projected by the monitor image data generated by the image processing controller are synthesized.

9. An image generating apparatus, comprising:

a real camera which captures camera image data projected onto a single real imaging plane and ranging from a close view to a distant view;

an image processing controller

which sets a virtual camera at a position higher than the real camera,

which sets a virtual stereoscopic projection plane, as a virtual projection plane from the virtual camera, including a close view screen set on ground and a distant view screen being connected to a side of the close view screen far from the real camera and set to have a predetermined upward tilt angle,

which sets a virtual stereoscopic imaging plane, as a virtual imaging plane of the virtual camera, including a close view imaging plane parallel to the close view screen and a distant view imaging plane connected to a side of the close view imaging plane near from the real camera and set to have a predetermined downward tilt angle,

which performs a coordinate transformation between a position of each picture element of the single camera image data captured by the real camera and a position of each picture element on the virtual imaging plane of the virtual camera through the virtual stereoscopic projection plane,

which transfers the each picture element of the camera image data onto the virtual stereoscopic imaging plane of the virtual camera according to the coordinate transformation, and

which generates monitor image data for obtaining a monitor image, the monitor image being as an image to be projected on the virtual stereoscopic imaging plane when the virtual stereoscopic projection plane is viewed from the virtual camera; and

a monitor which displays a synthesis image in which a downward image of the close view and a perspective image of an intermediate view to the distant view projected by the monitor image data generated by the image processing controller are synthesized.

10. An image generating method, comprising:

capturing camera image data projected onto a single real imaging plane of a real camera and ranging from a close view to a distant view;

setting a virtual camera at a position higher than the real camera;

setting a virtual stereoscopic projection plane, as a virtual projection plane, including a close view screen set on ground and a distant view screen being connected to a side of the close view screen far from the real camera and set to have a predetermined upward tilt angle;

performing a coordinate transformation between a position of each picture element of the single camera image data captured by the real camera and a position of each picture element on a virtual imaging plane of the virtual camera through the virtual stereoscopic projection plane;

generating monitor image data for obtaining a monitor image, the monitor image being as an image to be projected on the virtual imaging plane when the virtual stereoscopic projection plane is viewed from the virtual camera, by transferring the each picture element of the camera image data onto the virtual imaging plane of the virtual camera according to the coordinate transformation; and

displaying on a monitor a synthesis image in which a downward image of the close view and a perspective image of an intermediate view to the distant view projected by the generated monitor image data are synthesized.

11. The image generating method according to claim 10, wherein

a mapping table is generated by performing a coordinate transformation of a position of each picture element in an order of:

designating one position of the picture element on a virtual imaging plane of the virtual camera;

determining a first corresponding position on a virtual projection plane, corresponding to the designated one position of the picture element; and

determining a second corresponding position on the imaging plane of the real camera, corresponding to the first corresponding position, and

wherein the method performs a coordinate transformation, by using the mapping table, between the position of the each picture element of the single camera image data captured by the real camera and the position of the each picture element on the virtual imaging plane of the virtual camera.

12. The image generating method according to claim 10,

wherein a transformation relationship characteristic indicating a relationship between a distance of a picture element from an optical axis by a distorted image of the camera image data and a distance of a picture element from an optical axis by a non-distorted image of the camera image data is determined in advance, and

wherein the method corrects a distortion of a coordinate system of the position of the each picture element of the single camera image data captured by the real camera, by using the distance of the each picture element from the optical axis and the transformation relationship characteristic, to transform it into a coordinate system of the virtual imaging plane.

13. The image generating method according to claim 10, further comprising

setting the virtual camera on an arbitrary spatial position in accordance with an external operation to a virtual camera setting operation member.

14. The image generating method according to claim 10, further comprising

setting an upward tilt angle of a screen for the distant view to a screen for the close view in accordance with an external operation to a virtual stereoscopic projection plane setting operation member.

15. The image generating method according to claim 10, further comprising

setting a downward tilt angle of an imaging plane for the distant view to an imaging plane for the close view in accordance with an external operation to a virtual stereoscopic imaging plane setting operation member.

16. The image generating method according to claim 10, wherein the camera image data is captured by a rear camera attached to a rear part of a vehicle and which images a view behind the vehicle.

17. An image generating method, comprising:

capturing camera image data projected onto a single real imaging plane of a real camera and ranging from a close view to a distant view;

setting a virtual camera at a position higher than the real camera;

setting a virtual stereoscopic projection plane, as a virtual projection plane from the virtual camera, including a close view screen set on ground and a distant view screen being connected to a side of the close view screen far from the real camera and set to have a predetermined upward tilt angle;

setting a virtual stereoscopic imaging plane, as a virtual imaging plane of the virtual camera, including a close view imaging plane parallel to the close view screen and a distant view imaging plane connected to a side of the close view imaging plane near from the real camera and set to have a predetermined downward tilt angle;

performing a coordinate transformation between a position of each picture element of the single camera image data captured by the real camera and a position of each picture element on the virtual stereoscopic imaging plane of the virtual camera through the virtual stereoscopic projection plane;

generating monitor image data for obtaining a monitor image, the monitor image being as an image to be projected on the virtual stereoscopic imaging plane when the virtual stereoscopic projection plane is viewed from the virtual camera, by transferring the each picture element of the camera image data onto the virtual stereoscopic imaging plane of the virtual camera according to the coordinate transformation; and

displaying on a monitor a synthesis image in which a downward image of the close view and a perspective image of an intermediate view to the distant view projected by the generated monitor image data are synthesized.