Three-dimensional scope system with a single camera for vehicles

• US 6,172,601 B1
• Filed: 11/15/1999
• Issued: 01/09/2001
• Est. Priority Date: 11/26/1998
• Status: Expired due to Term

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Takahashi, Masami, Wada, Jyoji
• Primary Examiner(s): Hofsass, Jeffery A.
• Assistant Examiner(s): Nguyen, Phung
• Application Number:
  US09/440,139
• Time in Patent Office: 421 Days
• Field of Search: 340/436, 340/435, 340/903, 340/937, 348/135, 348/148, 348/149
• US Class Current: 340/436
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 1/31 providing stereoscopic vision
• CPC Class Codes: B60R 2300/107 using stereoscopic cameras
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/16 Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/296 : Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/30 : Image reproducers optical systems for producing stereoscopic or other three-dimensional effects G02B30/00
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0096 : Synchronisation or controlling aspects

Abstract

A single video camera is installed on a vehicle body. Two still pictures of an obstacle are obtained at different vehicle traveling positions. A picture memory stores the obtained still pictures. A feature point extracting section extracts at least one feature point from the data of the stored still pictures. A three-dimensional coordinate data calculating section calculates the distance from the vehicle body to the feature point and the height of the feature point based on the positional data of the feature point on the screen and the vehicle traveling distance data. A stereoscopic picture forming section forms a stereoscopic picture image of a detected obstacle based on three-dimensional coordinate data of the feature points. A monitor displays a plan view or a side view showing the relationship between the vehicle body and the obstacle, thereby allowing a driver to directly grasp the distance of the obstacle. When the obstacle is located within a predetermined distance from the vehicle body, the driver is cautioned against a predictable collision by the alarm sound or related image.

Claims

1. A three-dimensional scope system with a single camera for vehicles, said system comprising:

a single camera mounted on a vehicle body for successively obtaining two still pictures of a predetermined target at different vehicle traveling positions;

a means for obtaining a vehicle traveling distance between said different vehicle traveling positions;

a means for obtaining a positional change amount of at least one objective point of said target on a camera screen; and

a means for obtaining three-dimensional coordinate data of said objective point based on the vehicle traveling distance and said positional change amount of the objective point on the screen.

2. The three-dimensional scope system with a single camera in accordance with claim 1, further comprising:

a means for forming a stereoscopic picture image based on said three-dimensional coordinate data of said objective point; and

a monitor for displaying a resultant stereoscopic picture image.

3. The three-dimensional scope system with a single camera in accordance with claim 2,

wherein said means for forming said stereoscopic picture image includes a means for changing a viewpoint of the stereoscopic image to obtain an image viewing the predetermined target from an arbitrary position which is different from a viewpoint of said camera at the vehicle traveling positions.

4. The three-dimensional scope system with a single camera in accordance with claim 1, further comprising a means for generating an alarm when an obstacle is detected within a predetermined distance from said vehicle body.

5. A processing method for a three-dimensional scope system with a single camera mounted on a vehicle body, said method comprising:

a step of causing said single camera to successively obtain two still pictures of a predetermined target at different vehicle traveling positions;

a step of obtaining a positional change amount of at least one objective point of said target on a camera screen;

a step of obtaining three-dimensional coordinate data of said objective point based on a vehicle traveling distance between said different vehicle traveling positions and also based on said positional change amount of the objective point on the screen; and

a step of forming a stereoscopic picture image based on said three-dimensional coordinate data of said objective point.

6. A three-dimensional scope system with a single camera for vehicles, said system comprising:

a single camera mounted on a vehicle body for successively obtaining two still pictures of a predetermined target at different vehicle traveling positions, with an optical axis of said camera being fixed at a predetermined angle;

a means for obtaining a vehicle traveling distance between said different vehicle traveling positions;

a means for obtaining a positional change amount of at least one objective point of said target on a camera screen based on data obtained through image analysis applied to said two still pictures taken by said camera; and

a means for obtaining three-dimensional coordinate data of said objective point based on the vehicle traveling distance and said positional change amount of the objective point on the screen.

7. The three-dimensional scope system with a single camera in accordance with claim 6, further comprising:

a means for forming a stereoscopic picture image, modified from said still pictures taken by said camera, based on said three-dimensional coordinate data of said objective point; and

a monitor for displaying a resultant stereoscopic picture image.

8. The three-dimensional scope system with a single camera in accordance with claim 7, wherein said means for forming said stereoscopic picture image includes a means for changing a viewpoint to an arbitrary position different from a viewpoint of said camera.

9. The three-dimensional scope system with a single camera in accordance with claim 6, further comprising a means for generating an alarm when an obstacle is detected within a predetermined distance from said vehicle body as a result of a distance measurement based on said three-dimensional coordinate data.

12. The three-dimensional scope system with a single camera in accordance with claim 8, wherein said arbitrary position includes an infinite point or a driver'"'"'s position of a vehicle.

13. The three-dimensional scope system with a single camera in accordance with claim 8, wherein said arbitrary position is selected to obtain a picture image corresponding to at least one of a plan view and a side view of said vehicle body based on said three-dimensional coordinate data.

10. A processing method for a three-dimensional scope system with a single camera mounted on a vehicle body, said method comprising:

a step of causing said single camera to successively obtain two still pictures of a predetermined target at different vehicle traveling positions, while fixing an optical axis of said camera at a predetermined angle;

a step of obtaining a positional change amount of at least one objective point of said target on a camera screen based on data obtained through image analysis applied on said two still pictures taken by said camera;

a step of obtaining three-dimensional coordinate data of said objective point based on a vehicle traveling distance between said different vehicle traveling positions and also based on said positional change amount of the objective point on the screen; and

a step of forming a stereoscopic picture image based on said three-dimensional coordinate data of said objective point.

11. The processing method for a three-dimensional scope system with a single camera in accordance with claim 10, further comprising a step of generating the alarm when an obstacle is detected within a predetermined distance from said vehicle body as a result of a distance measurement based on said three-dimensional coordinate data.

14. The processing method for a three-dimensional scope system with a single camera in accordance with claim 10, further comprising a step of changing a viewpoint to an arbitrary position different from a viewpoint of said camera.

15. A three-dimensional scope system for vehicles, comprising:

a single camera mounted on a vehicle body for successively obtaining two still pictures of a predetermined target at different vehicle traveling positions, with an optical axis of said camera being fixed at a predetermined angle;

an image processing apparatus receiving said two still pictures from said single camera for analyzing data of the received still pictures to form a stereoscopic picture image; and

a monitor connected to said image processing apparatus to display the stereoscopic picture image produced from said image processing apparatus,

wherein

said image processing apparatus produces three-dimensional coordinate data of an objective point of said target based on a vehicle traveling distance between said different vehicle traveling positions and a positional change amount of said objective point on a camera screen based on data obtained through image analysis applied on said two still pictures taken by said camera.

16. The three-dimensional scope system in accordance with claim 15, wherein said image processing apparatus comprises:

a picture memory for storing digital data of said still pictures received from said camera;

a feature point extracting section for extracting said objective point of said target from said still pictures based on the digital data stored in said picture memory;

a three-dimensional coordinate data calculating section for calculating a position of said objective point in a three-dimensional coordinate system; and

a stereoscopic picture forming section for forming an arbitrary view of said target based on three-dimensional coordinate data obtained by said three-dimensional coordinate data calculating section.

17. The three-dimensional scope system in accordance with claim 16, wherein said stereoscopic picture forming section includes a viewpoint changing section for forming said arbitrary view from an arbitrary viewpoint different from a viewpoint of said camera.

18. The three-dimensional scope system in accordance with claim 15, further comprising a control unit for performing data communication with said image processing apparatus and activating a warning device as a result of a collision prediction based on said three-dimensional coordinate data.

19. A processing method for a three-dimensional scope system comprising:

a step of causing a single camera mounted on a vehicle body to successively perform first and second image pickup operations at a predetermined time interval Δ

t to obtain first and second still pictures of a predetermined target at different vehicle traveling positions, while fixing an optical axis “

C”

of said camera at a predetermined angle θ

0;

a step of storing data of said first and second still pictures;

a step of calculating a vehicle traveling distance “

I”

by multiplying a vehicle traveling speed “

v”

with said time interval Δ

t;

a step of extracting at least one feature point of said target from each of said first and second still pictures; and

a step of calculating three-dimensional coordinate data representing said feature point by using the following equations,

20. The processing method for a three-dimensional scope system in accordance with claim 19, further comprising a step of changing a viewpoint to an arbitrary position different from a viewpoint of said camera.

21. The processing method for a three-dimensional scope system in accordance with claim 19, further comprising a step of activating a warning device as a result of a distance measurement based on said three-dimensional coordinate data.