Surveillance system

• US 5,774,569 A
• Filed: 12/10/1996
• Issued: 06/30/1998
• Est. Priority Date: 07/25/1994
• Status: Expired due to Fees

• Current Assignee: H. Eugene W. Waldenmaier
• Original Assignee: H. Eugene W. Waldenmaier
• Inventors: Waldenmaier, H. Eugene W.
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US08/762,927
• Time in Patent Office: 567 Days
• Field of Search: 382/100, 382/103, 382/104, 348/148, 348/149, 348/143, 348/146, 364/436, 364/437
• US Class Current: 382/100
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors

Abstract

A Single optical element that also provides the structural support between optical elements, the alignment stability between optical elements, the weatherized window (environmental enclosure), and minimizes, or eliminates, central obscuration for a distortion free, wide field of view (a panoramic ±80°) optical surveillance system is disclosed. Simultaneously, the logic for a learn mode eliminates the need for programming "the location of detection zones" in the data processing system e.g. for a traffic surveillance and control systems, during or after installation of the system or for the off-line data processing. The synergism of these features enables complete real-time, e.g. within seconds after learn mode adaptation, traffic analysis and reporting of an entire multi-lane, omni-directional, non-orthogonal, traffic intersection, traffic circle or roadway from a single video surveillance system with no or minimal, programming knowledge on the part of the system installer or operator.

Claims

1. An adaptive learn mode method applied to the digital, time tagged, video data of a traffic surveillance system which eliminates or minimizes the need for operator intervention during setup or use of the surveillance system and the surveillance system'"'"'s data processing, wherein the adaptive learn mode method comprises a set of steps to establish and update functions that scale and align a coordinate system defined in object space to that of the coordinate system defined in surveillance camera'"'"'s pixel space, or image plane, so that a-priori limits, boundaries, images and criteria established in object space may be applied to the acquired data in pixel space, wherein the said adaptive learn mode method comprises:

a. executing one set of steps if installation site-specific data files are present;

b. executing a different set of steps if installation site-specific data files are not present and the data processing determines that the parallelism of object motion, or traffic lanes, are within a specified limit, i.e., the slopes of all best fit matrix equations are within the equivalent of some angle implying that an intersection is not present; and

c. executing a different set of steps if installation site-specific data files are not present and the data processing determines that the parallelism of object motion, or traffic lanes, exceeds a specified limit, i.e., the slopes of all best fit matrix equations equals or exceeds the equivalent of some angle implying that a standard intersection is present.

2. An adaptive learn mode method, according to claim 1, wherein the said adaptive learn mode method has determined to require site-specific data files in order to scale and align a coordinate system defined in object space to that of a coordinate system defined in pixel space so that a-priori limits, boundaries, images and criteria established in object space may be applied to the acquired data in pixel space, the said adaptive learn mode method comprising:

a. steps for verifying the presence of four site-specific data files comprising;

i. a file for predicted traffic patterns, comprising a matrix of matrix equations, herein identified as matrix_traffic to define predicted traffic centerlines as determined by traffic surveys or engineering studies, each matrix equation being unique to a specific lane and specified in a standard three dimensional coordinate system common to all object space information for the system, said coordinate system being based upon application site drawings with the origin specified as the intersect of the object surface and the axis of the controlling surveillance sensor or other fixed feature, the major axis being aligned parallel to the primary flow of traffic;

ii. a file for optimum location and shape of detection zones, comprising a matrix of matrix equations, herein identified as matrix_zone, to define each boundary of ideally located detection zones that are determined by traffic surveys or engineering studies, each matrix equation being specified in the standard coordinate system common to all object space information for the system;

iii. a file for installation error correction, comprising a matrix, herein identified as matrix_optic, to define the ideal optical scaling between object space and pixel space for the system installation with no positional or angular installation errors;

iv. a file for installation error correction, comprising a matrix, herein identified as matrix_install, to define the correction sensitivities for matrix_optic for a unit of translational and angular error in surveillance sensor placement or alignment, the amplitude of the correction sensitivities being defined by a matrix, herein identified as matrix_weight ;

b. steps to compute a matrix of matrix equations, herein identified as matrix_observed from the corrected centroids or vectors in the time tagged data in pixel coordinates comprising;

i. the corrected centroids defined as the standard mathematical centroids, or other computed values, of derived images from objects that are nominally the height of a standard vehicle, the departure of an object'"'"'s height from that of a standard height providing a shift in width and length in a known direction as a function of field position, whereby when an image appears wider than a standard width, which is specified as a function of field position, the centroid is compensated to the height of the standard, and the date file for that image is updated with the object height for subsequent processing, and where the corrected centroid is the standard mathematical centroid, or other computed value, when standard width is not specified as a function of field position;

ii. a matrix equation, for matrix_observed, is fitted to each traffic lane by a set or regression rules applied to the sorted time tagged corrected centroids, said regression rules rejecting the data that is not within a specified confidence limit and rejecting equations that have insufficient data;

iii. a matrix equation, for matrix_observed, may be optionally fitted to each traffic lane by a set of regressions rules applied to image vectors from the same derived image data sets;

c. steps to compute the transfer function, herein identified as matrix_scale, that sets matrix_traffic equal to, or at optimized correlation with, matrix_observed by the periodic refinement of matrix_weight ;

space="preserve" listing-type="equation">matrix.sub.scale ≅

f(matrix.sub.observed, matrix.sub.traffic)≅

g(matrix.sub.optic, matrix.sub.weight matrix.sub.install);

d. steps to align and scale the ideal detection zones defined in object space into pixel space by applying the matrix product of matrix_scale to each matrix equation of matrix_zone defining a new matrix, herein identified as matrix_final, which are the pixel space boundaries to define the acceptance limits for corrected centroids or vectors, that are equivalent to objects within the bounds of the detection zones on the ground; and

e. steps to identify when an object of interest, the corrected centroid of vector of the processed image, is within the boundaries of a detection zone, where said steps comprise;

i. setting a true state for each boundary equation of matrix_final when the image corrected centroid or vector satisfies that applicable boundary condition and setting a true state for each zone when each boundary condition for that zone is verified true, or

ii setting a true state for each zone of matrix_final when the coordinate of an image corrected centroid or vector matches a value in a look-up table which establishes the correlation of all pixels located within the appropriate boundaries of that said zone.

3. An adaptive learn mode method, according to claim 1, wherein the said adaptive learn mode method has determined both that installation site-specific data files are not required and that an intersection is not present, as determined by the parallelism of traffic lanes being within a specified limit, i.e., the slopes of all matrix_observed equations are within the equivalent of some angle implying that an intersection is not present, the said adaptive learn mode method will implement the following in order to apply a-priori object space limits;

boundaries, images and criteria to the acquired data in pixel space comprising;

a. steps to compute a matrix of matrix equations, herein identified as matrix_observed, from the corrected centroids or vectors in the time tagged data in pixel coordinates comprising;

i. the corrected centroids defined as the standard mathematical centroids, or other computed values, of derived images from objects that are nominally the height of a standard vehicle, the departure of an object'"'"'s height from that of a standard height providing a shift in width and length in a known direction as a function of field position, whereby when an image appears wider than a standard width, which is specified as a function of field position, the centroid is compensated to the height of the standard, and the data file for that image is updated with the object height for subsequent processing, and where the corrected centroid is the standard mathematical centroid, or other computed value, when standard width is not specified as a function of field position;

ii. a matrix equation, for matrix_observed, is fitted to each traffic lane by a set of regression rules applied to the sorted time tagged corrected centroids, these regression rules rejecting the data that is not within a specified confidence limit and rejecting equations that have insufficient data; and

iii. a matrix equation, for matrix_observed, may be optionally fitted to each traffic lane by a set of regression rules applied to image vectors from the same original derived image data set;

b. steps to align and scale the emulated detection zones of object space into pixel space comprising;

i. lane widths defined by the difference between adjacent traffic centerlines, independent of traffic direction, where each centerline is defined by a unique matrix equation of matrix_observed ;

ii. detection zone boundaries between adjacent lanes, herein identified as a matrix equation in matrix_zone, defined by the average of the two lane centerlines where each centerline is defined by a unique matrix equation in matrix_observed ;

iii. detection zone boundaries to the side of a lane with no adjoining lane, herein identified as a matrix equation in matrix_zone, defined by the lane'"'"'s centerline defined by a unique matrix equation in matrix_observed plus, or minus, one half the lane'"'"'s width as established above in (b)(i);

iv. detection zone boundaries, herein identified as matrix equations in matrix_zone, defining the zone'"'"'s length and location along each lane'"'"'s centerline defined by a matrix equation normal to the matrix equation of the lane'"'"'s centerline as defined in matrix_observed and placed at an equivalent object distance, i.e., three-and-one-half (31/2) lane widths, between boundaries starting near and symmetric about the optical system'"'"'s centerline;

v. detection zone encoding being modified by a look-up table to maximize the number of detection zones consistent with the number of lanes and a specified maximum number of detection zones, i.e. 32 identifiers;

vi. when site specific files are not present, matrix_scale is unity and matrix_zone equals matrix_finial ; and

c. steps to identify when an object of interest, the corrected centroid or vector of the processed image, is within the boundaries of a detection zone in pixel space, wherein said steps comprise;

i. setting a true state for each boundary equation of matrix_final when the image corrected centroid satisfies that applicable boundary condition and setting a true state for each zone when each boundary condition for that zone is verified true, or

ii. setting a true state for each zone in matrix_final when the coordinate of an image corrected centroid or vector matches a value in a look-up table which establishes the correlation of all pixels located within the appropriate boundaries of that said zone.

4. An adaptive learn mode method, according to claim 1, wherein the said adaptive learn mode method has determined both that installation site-specific data files are not required and that an intersection is present, as determined by the parallelism between traffic lanes being equal to or exceeding a specified limit, i.e., the slope between certain matrix_observed equations are equal to or greater than the equivalent of some angle implying that an intersection is present, the said adaptive learn mode method will implement the following in order to apply a-priori object space limits, boundaries, images and criteria to the acquired data in pixel space comprising:

a. steps to compute a matrix of matrix equations, herein identified as matrix_observed, from the corrected centroids or vectors in the time tagged data in pixel coordinates comprising;

i. the corrected centroids defined as the standard mathematical centroids, or other computed values, of derived images from objects that are nominally the height of a standard vehicle, the departure of an object'"'"'s height from that of a standard height providing a shift in width and length in a known direction as a function of field position, whereby when an image appears wider than a standard width, which is specified as a function of field position, the centroid is compensated to the height of the standard, and the data file for that image is updated with the object height for subsequent processing, and where the corrected centroid is the standard mathematical centroid, or other computed value, when standard width is not specified as a function of field position;

ii. a matrix equation, for matrix_observed, is fitted to each traffic lane by a set of regression rules applied to the sorted time tagged corrected centroids, these regression rules rejecting the data that is not within a specified confidence limit and rejecting equations that have insufficient data; and

iii. a matrix equation, for matrix_observed, may be optionally fitted to each lane that is defined by image vectors from the same derived image data set;

b. steps to align and scale the detection zones of object space into pixel space comprising;

i. lane widths defined by the difference between adjacent traffic centerlines, independent of traffic direction, where each centerline is defined by a unique matrix equation of matrix_observed ;

ii. detection zone boundaries between adjacent lanes, herein identified as a matrix equation in matrix_zone, defined by the average of the two lane centerlines where each centerline is defined by a unique matrix equation in matrix_observed ;

iii. detection zone boundaries to the side of a lane with no adjoining lane, herein identified as a matrix equation in matrix_zone, defined by the lane'"'"'s centerline defined by a unique matrix equation in matrix_observed plus, or minus, one half the lane'"'"'s width as established above in b) i);

iv. detection zone boundaries, herein identified as matrix equations in matrix_zone, defining the zone'"'"'s length and location along each lane'"'"'s centerline, i.e., three-and-one-half (31/2) lane'"'"'s width between adjacent boundaries, with said boundary defined by matrix equations normal to the matrix equation of the lane'"'"'s centerline as defined in matrix_observed, and with the first boundary for each said lane at a length, i.e. one-half of a lane'"'"'s width, from the lane'"'"'s reference, wherein the said lane'"'"'s reference is defined as the intercept coordinate of the said lane'"'"'s centerline and other traffic lane centerlines as determined by a matrix application of the constants of matrix_observed, and also as the intercept coordinate along the direction of traffic closest to the maximum of centroids per unit lane length, where this maximum is due to slowing and static objects in queue for a red light or stop sign; and

c. steps to identify when an object of interest, the corrected centroid or vector of the processed image, is within the boundaries of a detection zone wherein said steps comprise;

i. setting a true state for each boundary equation of matrix_final when the image corrected centroid or vector satisfies that applicable boundary condition and setting a true state for each zone when each boundary condition for that zone is verified true, or

ii. setting a true state for each zone of matrix_final when the coordinate of an image corrected centroid or vector matches a value in a look-up table which establishes the correlation of all pixels located within the appropriate boundaries of that said zone.

5. An adaptive learn mode method, according to claim 1, wherein the said adaptive learn mode method provides the steps for automated detection zone identification and statistical reporting wherein the said adaptive learn mode comprises:

a. sorting the matrix equations of matrix_finial to uniquely pre identify and encode each detection zone, comprising;

i. labeling each lane and it'"'"'s detection zones as either an approach to or departure from the lane'"'"'s intersection or the geometric center if an intersection is not present;

ii. sequentially labeling each lane, and zone in each lane, by established criteria and pattern so that all zone related data that is so labeled will be uniquely and properly identified by the applying the said established criteria and pattern to the reported data;

iii. implementing a-priori zone reporting sequence for each established criteria and pattern and thereby eliminating the requirement to attach a unique zone identifier to the reported data stream; and

b. reporting the look up table parameters associated with each detection zone, which parameters for each detection zone may include the current number of corrected centroids within the zone, the total number of centroids per unit time, the average centroid rate of motion, vehicle classification, and other definable statistics wherein the reported values may also be the change in absolute values with the absolute value only reported on an infrequent basis.

6. A refractive optical element of a catadioptric optical system comprising at least one refractive optical element and at least one mirror, wherein the said refractive optical element provides the structural support and alignment stability between other optical elements and a mirror, and also provides the protective environmental enclosure of the optical elements and the mirror, and introduces no obscuration and/or vignetting of images for the field of view about the optical axis, and where one surface of the refractive optical element is the last surface, excluding films and coating surfaces, of the optical system, and where the said refractive optical element has the form where the radial distances from the optical axis to the inner and outer surfaces are symmetrical about the optical axis and varies as function of axial position to form a walled cylinder or the partial shell of a truncated cone, sphere, or asphere.

7. A refractive optical element of a catadioptric optical system comprising at least one refractive optical element and at least one mirror, according to claim 6, wherein the said refractive optical element provides the structural support and alignment stability between other optical elements and a mirror, and also provides the protective environmental enclosure of the optical elements and the mirror, and introduces no obscuration and/or vignetting of images for the field of view about the optical axis, and where one surface of the refractive optical element is the last surface, excluding films and coating surfaces, of the optical system, and where the refractive optical element has the form where the radial distances from the optical axis to the inner and outer surfaces are symmetrical about the optical axis and varies as function of axial position to also form a lens element that extends or modifies the field of view or spatial scaling of the system, e.g., may reduce or eliminate central obscuration.

8. A solid, three optical surface, catadioptric element of an optical system, wherein the said catadioptric element provides the structural support and alignment stability between other optical elements and a mirror of the said optical system, and also provides the protective environmental enclosure of the optical elements and the mirror, and introduces no obscuration and/or vignetting of images for the field of view about the optical axis and where one surface of said catadioptric element is the last surface, excluding films and coating surfaces, of the optical system, and where the said catadioptric element'"'"'s three effective surfaces listed sequentially from the focal plane side, comprise:

a. a first surface, normal to and symmetric about the optical axis, primarily optimized for aberration correction, focus control and secondarily for field angle magnification between image and object space;

b. a second surface which is a mirror normal to and symmetric about the optical axis and optimized as the primary transfer function between radial distance in the focal plane and the object surface and with the mirror surface redirecting the rays between the first surface and a third surface in accordance with basic reflection equations where the reflected and incident angles are of equal magnitude relative to the instantaneous surface normal; and

c. a third surface, which is part of the physical surface connecting the first and the second optical surfaces, which is nominally parallel with, and a radial function of, the axial distance along the optical axis and is optimized as the secondary transfer function between radial distance in the focal plane and the object surface, said third surface redirecting the rays between the second surface of the catadioptric element and the object surface in accordance with Snell'"'"'s basic refraction equation of n₁ Sin θ

₁ =n₂ Sin θ

₂, where n is the media'"'"'s index of refraction, θ

is the ray angle relative to the instantaneous surface normal and the subscripts 1 and 2 indicate the media of the ray, and the shape of this surface also provides modification for aberration correction and focus control.

9. A solid, four optical surface, catadioptric element of an optical system, wherein the said catadioptric element provides the structural support and alignment stability between other optical elements and a mirror of the said optical system, and also provides the protective environmental enclosure for the optical elements and the mirror, and introduces no obscuration and/or vignetting of images for the field of view about the optical axis and where one surface of said catadioptric element is the last surface, excluding films and coating surfaces, of the optical system, and where the said catadioptric element'"'"'s four optical surfaces listed sequentially from the focal plane side, comprises:

a. a first physical surface which is normal to and symmetric about the axis and has two optical surfaces herein identified as the first optical surface and third optical surface, where;

i. the first optical surface is a transparent central core and may be the system'"'"'s aperture stop at or near the vertex of a third optical surface;

ii. the third optical surface is a reflective aspheric surface, about the first surface, and is the second reflective element after the aperture stop, where this mirror is optimized as the primary transfer function between radial distance in the focal plane and the object surface, and this surface redirects the rays between the second optical surface, which is also a mirror, and a fourth optical surface in accordance with basic reflection equations where the reflected and incident angles are of equal magnitude relative to the instantaneous surface normal;

b. a second optical surface which is a mirror that is normal to and symmetric about the axial axis, and may be flat or optimized as either, or both, a low power aspheric or an axicon, where its primary function is to redirect the rays of the first optical surface to the third optical surface in accordance with basic reflection equations where the reflected and incident angles are of equal magnitude relative to the instantaneous surface normal; and

c. a fourth optical surface, which is part of the physical surface connecting the second and the third optical surfaces, and which is nominally parallel with, and a radial function of, the axial distance along the optical axis and optimized as a transfer function between radial distance in the focal plane and the object surface by redirecting the rays between the catadioptric element'"'"'s third optical surface and the object'"'"'s surface in accordance with Snell'"'"'s basic refraction equation of n₁ Sin θ

₁ =n₂ Sin θ

₂, where n is the media'"'"'s index of refraction, θ

is the ray angle relative to the instantaneous surface normal and the subscripts 1 and 2 indicate the media of the ray, and the shape of this surface also provides modifications for aberration correction and focus control.