Video technique for indicating moving objects from a movable platform

• US 5,473,364 A
• Filed: 06/03/1994
• Issued: 12/05/1995
• Est. Priority Date: 06/03/1994
• Status: Expired due to Term

• Current Assignee: Sarnoff Corporation (SRI International, Inc.)
• Original Assignee: David Sarnoff Research Center, Inc.
• Inventors: Burt, Peter J.
• Primary Examiner(s): Britton, Howard W.
• Application Number:
  US08/253,512
• Time in Patent Office: 550 Days
• Field of Search: 348/144, 348/154, 348/155, 348/47, 348/169
• US Class Current: 348/47
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/1963 Arrangements allowing camera rotation to change view, e.g. pivoting camera, pan-tilt and zoom [PTZ]
• CPC Class Codes: G08B 13/19641 Multiple cameras having overlapping views on a single scene
• CPC Class Codes: G08B 13/19647 Systems specially adapted for intrusion detection in or around a vehicle
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0085 Motion estimation from stereoscopic image signals
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

An imaging technique, suitable for surveillance use by a moving robotic vehicle carrying two spatially displaced fixed cameras aligned with the direction of travel or, alternatively, a single camera movable to either of two positions on the vehicle corresponding to the positions occupied by the fixed cameras, is capable in a moving object indicator mode of deriving first image data of a viewed scene at a first time from a first position with respect to the vehicle and of deriving second image data of the viewed scene at a second time from a second position with respect to the vehicle that is spatially displaced a given distance from the first position such that the spatial position with respect to the viewed scene depicted by the second image data at least approximates the spatial position with respect to the viewed scene depicted by the first image data. As an aid to detecting driving hazards encountered by the moving vehicle, the imaging technique may also be operated in a stereo-vision mode, in which the first and second image data is derived at the same time from the same angle of gaze of the viewed scene.

Claims

1. In apparatus for viewing a 3-dimensional scene from a moving platform, wherein said 3-dimensional scene may include objects at different distances from said moving platform;

said apparatus comprising;

means including camera means situated on said moving platform that points said camera means in a predetermined direction toward said scene for deriving first image data of said scene at a first earlier time from a first position with respect to said moving platform and that points said camera means in substantially the same predetermined direction toward said scene for deriving second image data of said scene at a second later time from a second position with respect to said moving platform that is spatially displaced a given distance from said first position such that the spatial position with respect to said viewed scene depicted by said second image data at least approximates the spatial position with respect to said viewed scene depicted by said first image data, whereby said first and second image data results in minimizing the depiction of parallax between relatively near and far objects in said 3-dimensional scene caused by movement of said moving platform; and

image processing means responsive to said first and second image data for deriving information therefrom that indicates those objects in said viewed scene that are moving with respect to stationary objects in said viewed scene and substantially removes any indication of said stationary objects in said viewed scene.

2. The apparatus defined in claim 1, wherein:

said moving platform is moving in a given direction of travel with a given velocity;

said means including camera means includes a camera situated at said first position at said earlier first time and a camera situated at said second position at said later first time, said second position being spatially displaced from said first position by a given distance that is substantially aligned with said given direction of travel; and

the time difference between said later second time and said earlier first time is substantially equal to the quotient of said given velocity divided by said given distance.

3. The apparatus defined in claim 2, wherein:

said means including camera means includes first and second separate cameras, said first camera being fixed to said moving platform at said first position and said second camera being fixed to said moving platform at said second position.

4. The apparatus defined in claim 2, wherein:

said means including camera means includes a single camera that is movable with respect to said movable platform between said first position and second positions, said single camera occupying said first position at said earlier time and occupying said second position at said later time.

5. The apparatus defined in claim 1, wherein:

the image frames are in analog format; and

said image processing means includes analog-to-digital converter means for converting each of said first and second image data from analog format to digital format.

6. The apparatus defined in claim 5, wherein:

said means including camera means derives said first image data and said second image data as image frames; and

said image processing means comprises frame store memory means responsive to a first digital output from said analog-to-digital converter means for time-delaying said first image data by the time difference between said later second time and said earlier first time, means including warp means and residual displacement estimation means for warping an output from said frame store memory means so that stationary objects depicted by said warped time-delayed first image data substantially coincide in position with stationary objects depicted by a second digital output that defines said second image data, means for computing and means, including subtraction means and a Gaussian pyramid, for computing the local change energy estimation in the difference between said warped time-delayed first image data and said second image data.

7. The apparatus defined in claim 6, wherein:

said means including camera means derives said first image data from a first camera and derives said second image data from a second camera; and

said analog-to-digital converter means comprises a first analog-to-digital converter for converting said first image data from analog format to digital format and a second analog-to-digital converter for converting said second image data from analog format to digital format.

8. The apparatus defined in claim 6, wherein:

said means including camera means derives said first image data from a single camera at said earlier first time and derives said second image data from the same single camera at said later second time; and

said analog-to-digital converter means comprises a single analog-to-digital converter for converting said first image data from analog format to digital format at said earlier first time and for converting said second image data from analog format to digital format at said later second time.

9. In apparatus comprising means including camera means situated on said moving platform for viewing a 3-dimensional scene from a moving platform, and image processing means responsive to image data from said means including camera means;

wherein;

said moving platform comprises a vehicle that is movable in a given direction of travel; and

said means including camera means comprises a bar oriented substantially in a horizontal plane that is rotatably mounted to said vehicle at a given point of said bar in between opposite ends thereof for rotation of said bar about a substantially vertical axis, first and second cameras respectively attached to said bar on either side of said given point toward the opposite ends of said bar with said first camera occupying said first position and said second camera occupying said second position.

10. The apparatus defined in claim 9, wherein:

said means including camera means further comprises means rotating said bar into a position that is in substantial alignment with said given direction of travel with said first camera being situated forward of said second camera with respect to said given direction of travel;

whereby said apparatus can be operated in a moving-object indicator mode in which said first camera derives said first image data of said scene at an earlier first time from said first position with respect to said vehicle and said second camera derives said second image data of said scene at a later second time from said second position with respect to said vehicle that is spatially displaced by said bar a given distance from said first position such that the spatial position with respect to said viewed scene depicted by said second image data at least approximates the spatial position with respect to said viewed scene depicted by said first image data.

11. The apparatus defined in claim 9, wherein:

said means including camera means further comprises means rotating said bar into a position that is angularly displaced from said given direction of travel by a given angle of gaze;

whereby said apparatus can be operated in a stereo-vision mode in which said first camera derives said first image data of said scene at said given angle of gaze from said first position with respect to said vehicle at the same time as said second camera derives said second image data of said scene at said given angle of gaze from said second position with respect to said vehicle.

12. An imaging method for distinguishing moving objects from stationary objects in a 3-dimensional scene that is viewed from a moving platform, wherein said 3-dimensional scene may include objects at different distances from said moving platform;

said method comprising the step of;

deriving first image data of said scene at a first time from a first position with respect to said moving platform by pointing camera means in a predetermined direction toward said scene and for deriving second image data of said scene at a second time from a second position with respect to said moving platform by pointing said camera means in substantially the same predetermined direction toward said scene, said second position being spatially displaced a given distance from said first position such that the spatial position with respect to said viewed scene depicted by said second image data at least approximates the spatial position with respect to said viewed scene depicted by said first image data;

whereby said first and second image data results in minimizing the depiction of parallax between relatively near and far objects in said 3-dimensional scene caused by movement of said moving platform.