Real-time video triggering for traffic surveillance and photo enforcement applications using near infrared video acquisition

• US 10,713,499 B2
• Filed: 04/23/2012
• Issued: 07/14/2020
• Est. Priority Date: 04/23/2012
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Inventors: Wu, Wencheng
• Primary Examiner(s): Pendleton, Brian T
• Assistant Examiner(s): Lotfi, Kyle M
• Application Number:
  US13/453,089
• Publication Number:
  US 20130278761A1
• Time in Patent Office: 3,004 Days
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A method and system for real-time video triggering for traffic surveillance and photo enforcement comprises receiving a streaming video feed and performing a spatial uniformity correction on each frame of the streaming video feed and resampling the video feed to a lower spatial resolution. Motion blobs are then detected. Next a three-layered approach is used to identify candidate motion blobs which can be output to a triggering module to trigger a video collection action.

Claims

1. A method for real-time video triggering comprising:

receiving a streaming video feed from a video camera with a non-transitory computer readable medium and a computer-usable data carrier storing instructions that, when executed by a computer, cause the computer to perform a function, wherein said streaming video feed comprises high-resolution near infrared streaming video filtered with a 870 nm band pass filter;

resampling said streaming video to a lower spatial resolution of said streaming video with a non-transitory computer readable medium and a computer-usable data carrier storing instructions that, when executed by a computer, cause the computer to perform a function, wherein each row of said streaming video is resampled by a factor of 8 and each column of said streaming video is resampled by a factor of 8;

detecting a plurality of motion blobs in said resampled video stream with a non-transitory computer readable medium and a computer-usable data carrier storing instructions that, when executed by a computer, cause the computer to perform a function, wherein detecting said plurality of motion blobs further comprises analyzing said resampled video and providing dimensional characteristics of said plurality of motion blobs, wherein said plurality of motion blobs are indicative of a target that should activate said triggering;

eliminating candidate motion blobs from said plurality of motion blobs as candidates for triggering a photo enforcement application with a non-transitory computer readable medium and a computer-usable data carrier storing instructions that, when executed by a computer, cause the computer to perform a function, according to a filtering approach comprising three ordered filtering layers, a first computationally inexpensive layer configured to eliminate candidate motion blobs according to size, a second layer configured to eliminate candidate motion blobs, not eliminated in said first layer, according to intensity, and a third layer configured to eliminate candidate motion blobs, not eliminated in said first layer and said second layer, according to a classifier;

outputting said remaining candidate motion blobs not eliminated in said first layer, said second layer, and said third layer; and

triggering said photo enforcement application according to said output of said remaining candidate motion blobs with a trigger associated with a non-transitory computer readable medium and a computer-usable data carrier storing instructions that, when executed by a computer, cause the computer to perform a function;

wherein detecting a plurality of motion blobs in said resampled video stream further comprises;

labeling an initial foreground and background in said resampled video stream; and

applying morphological filtering to said labeled video frame to identify a plurality of motion blobs in said resampled video stream wherein said dimensional characteristics of said plurality of motion blobs comprise motion blob mask dimensional characteristics, centroid dimensional characteristics, size, bounding box dimensional characteristics, and aspect ratio;

wherein detecting a plurality of motion blobs in said resampled video stream further comprises;

labeling an initial foreground and background in said resampled video stream; and

applying morphological filtering to said labeled video frame to identify a plurality of motion blobs in said resampled video stream wherein said dimensional characteristics of said plurality of motion blobs comprise motion blob mask dimensional characteristics, centroid dimensional characteristics, size, bounding box dimensional characteristics, and aspect ratio; and

wherein identifying candidate motion blobs from said plurality of motion blobs as candidates for triggering said photo enforcement application further comprises;

first eliminating any motion blob from said plurality of motion blobs in said resampled video stream as a candidate motion blob if a size and aspect ratio of said motion blob are not within a predefined size and aspect ratio range indicative of a vehicle license plate with said first computationally inexpensive layer configured to eliminate candidate motion blobs according to size;

second eliminating any motion blob from said remaining plurality of motion blobs at original resolution as a candidate motion blob if a mean intensity of said motion blob and a standard deviation of an intensity of said motion blob does not fit a predefined characteristic signature of a vehicle license plate with said second layer configured to eliminate candidate motion blobs, not eliminated in said first layer, according to intensity; and

third eliminating a motion blob from said remaining plurality of motion blobs at original resolution as a candidate motion blob if said motion blob is not identified as a candidate motion blob by a candidate motion blob classifier, wherein said motion blob classifier is a linear discriminate analysis license plate vs. tail light classifier operating on a histogram of gradient feature vectors, with said third layer configured to eliminate candidate motion blobs, not eliminated in said first layer and said second layer, according to a classifier.

2. The method of claim 1 wherein outputting said candidate motion blobs further comprises outputting a frame number.

3. The method of claim 2 wherein outputting said candidate motion blobs further comprises outputting a plurality of centroids.

4. The method of claim 1 wherein outputting said candidate motion blobs further comprises outputting a plurality of coordinates of a bounding box associated with each of said candidate motion blobs.

5. The method of claim 1 wherein outputting said candidate motion blobs further comprises:

outputting a frame number, a time stamp, a plurality of coordinates of centroids, and a plurality of coordinates of a bounding box associated with each of said candidate motion blobs.

6. The method of claim 5 wherein said candidate motion blobs comprise license plates.

7. The method of claim 1 wherein said streaming video feed comprises a traffic surveillance video feed.

8. The method of claim 1 further comprising:

performing a spatial uniformity correction on each frame of said streaming video feed wherein said spatial uniformity correction performed on each frame of said streaming video feed is only performed when an associated ambient light level is low and an artificial light source is used to illuminate said target.

9. A system for real-time video triggering comprising:

a video camera;

a computer system comprising a processor that communicates electronically with a memory, and a non-transitory computer-usable medium embodying computer program code stored in said memory, said non-transitory computer-usable medium capable of communicating with said processor, said computer program code comprising instruction modules executable by said processor;

a video receiving module for receiving a streaming video feed from said video camera, wherein said video camera is configured to collect high-resolution near infrared streaming video filtered with a 870 nm band pass filter;

a resampling module configured to receive said streaming video feed and resample said streaming video to a lower spatial resolution of said streaming video, wherein each row of said streaming video is resampled by a factor of 8 and each column of said streaming video is resampled by a factor of 8;

a detection module configured to detect a plurality of motion blobs in said resampled video stream wherein detecting said plurality of motion blobs further comprises analyzing said resampled video and providing dimensional characteristics of said plurality of motion blobs, wherein said plurality of motion blobs are indicative of a target that should activate said triggering;

an identification module configured to eliminate candidate motion blobs from said plurality of motion blobs as candidates for triggering a photo enforcement application according to a filtering approach comprising three ordered filtering layers, a first computationally inexpensive layer configured to eliminate candidate motion blobs according to size, a second layer configured to eliminate candidate motion blobs not eliminated in said first layer according to intensity, and a third layer configured to eliminate candidate motion blobs not eliminated in said first layer and said second layer according to a classifier;

an output module wherein said remaining candidate motion blobs not eliminated in said first layer, said second layer, and said third layer are provided to a triggering module configured to trigger a collection action;

wherein said detection module is further configured to;

label an initial foreground and background in said resampled video stream; and

apply morphological filtering to said labeled video frame to identify a plurality of motion blobs in said resampled video stream wherein said dimensional characteristics of said plurality of motion blobs comprise motion blob mask dimensional characteristics, centroid dimensional characteristics, size, bounding box dimensional characteristics, and aspect ratio;

wherein said detection module is further configured to;

label an initial foreground and background in said resampled video stream; and

apply morphological filtering to said labeled video frame to identify a plurality of motion blobs in said resampled video stream wherein said dimensional characteristics of said plurality of motion blobs comprise motion blob mask dimensional characteristics, centroid dimensional characteristics, size, bounding box dimensional characteristics, and aspect ratio; and

wherein said identification module is further configured to;

first eliminate any motion blob from said plurality of motion blobs in said resampled video stream as a candidate motion blob if a size and aspect ratio of said motion blob are not within a predefined size and aspect ratio range indicative of a vehicle license plate with said first computationally inexpensive layer configured to eliminate candidate motion blobs according to size;

second eliminate any motion blob from said remaining plurality of motion blobs at original resolution as a candidate motion blob if a mean intensity of said motion blob and a standard deviation of an intensity of said motion blob does not fit a predefined characteristic signature of a vehicle license plate with said second layer configured to eliminate candidate motion blobs, not eliminated in said first layer, according to intensity; and

third eliminate a motion blob from said remaining plurality of motion blobs at original resolution as a candidate motion blob if said motion blob is not identified as a candidate motion blob by a candidate motion blob classifier, wherein said motion blob classifier is a linear discriminate analysis license plate vs. tail light classifier operating on a histogram of gradient feature vectors, with said third layer configured to eliminate candidate motion blobs, not eliminated in said first layer and said second layer, according to a classifier.

10. The system of claim 9 wherein outputting of said candidate motion blobs by said output module comprises outputting a frame number.

11. The system of claim 9 wherein outputting of said candidate motion blobs by said output module comprises outputting a plurality of centroids.

12. The system of claim 9 wherein outputting of said candidate motion blobs by said output module further comprises outputting a plurality of coordinates of a bounding box associated with each of said candidate motion blobs.

13. The system of claim 9 wherein said output module is further configured to:

output a frame number, a time stamp, a plurality of coordinates of centroids, and a plurality of coordinates of a bounding box associated with each of said candidate motion blobs to a triggering module.

14. The system of claim 13 wherein said candidate motion blobs comprise license plates.

15. The system of claim 9 wherein said streaming video feed comprises a traffic surveillance video feed.

16. The system of claim 9 further comprising a spatial uniformity correction module wherein a spatial uniformity correction is applied to said streaming video feed and said spatial uniformity correction preformed on each frame of said streaming video feed is only performed when an associated ambient light level is low and an artificial light source is used to illuminate said target.

17. An apparatus comprising:

a video camera for collecting streaming video; and

a computer program product comprising;

a non-transitory computer readable medium and a computer-usable data carrier storing instructions that, when executed by a computer, cause the computer to perform a method of real-time video triggering, the method comprising;

receiving a streaming video feed from said video camera;

resampling said streaming video to a lower spatial resolution of said streaming video, wherein each row of said streaming video is resampled by a factor of 8 and each column of said streaming video is resampled by a factor of 8;

detecting a plurality of motion blobs in said resampled video stream wherein detecting said plurality of motion blobs further comprises analyzing said resampled video and providing dimensional characteristics of said plurality of motion blobs, wherein said plurality of motion blobs are indicative of a target that should activate said triggering by labeling an initial foreground and background in said resampled video stream;

applying morphological filtering to said resampled video stream to identify a plurality of motion blobs in said resampled video stream;

providing dimensional characteristics of said plurality of motion blobs comprising motion blob mask dimensional characteristics, centroid dimensional characteristics, size, bounding box dimensional characteristics, and aspect ratio according to a three layered filtering approach comprising;

first eliminating any motion blob from said plurality of motion blobs in said resampled video stream as a candidate motion blob if a size and aspect ratio of said motion blob are not within a predefined size and aspect ratio range indicative of a vehicle license plate;

next eliminating any motion blob from said remaining plurality of motion blobs at original resolution as a candidate motion blob if a mean intensity of said motion blob and a standard deviation of an intensity of said motion blob does not fit a predefined characteristic signature of a vehicle license plate;

next eliminating a motion blob from said remaining plurality of motion blobs at original resolution as a candidate motion blob if said motion blob is not identified as a candidate motion blob by a candidate motion blob classifier video stream as candidates for triggering photo enforcement application action, wherein said motion blob classifier is a linear discriminate analysis license plate vs. tail light classifier operating on a histogram of gradient feature vectors;

outputting a plurality of coordinates of centroids and a bounding box associated with said candidate motion blobs not eliminated in said first layer, said second layer, and said third layer; and

triggering said photo enforcement application with a trigger.

18. The computer program product of claim 17, wherein said streaming video feed comprises high-resolution near infrared streaming video filtered with a 870 nm band pass filter.

19. The computer program product of claim 18, wherein said candidate motion blobs comprise license plates.

20. The method of claim 18, wherein said spatial uniformity correction preformed on each frame of said streaming video feed is only performed when an associated ambient light level is low and an artificial light source is used to illuminate the target.