Automated video surveillance systems

• US 9,313,463 B2
• Filed: 06/09/2010
• Issued: 04/12/2016
• Est. Priority Date: 06/09/2009
• Status: Active Grant

• Current Assignee: Wayne State University
• Original Assignee: Wayne State University
• Inventors: Sarhan, Nabil J.
• Primary Examiner(s): Rao, Andy
• Application Number:
  US13/376,392
• Publication Number:
  US 20120133774A1
• Time in Patent Office: 2,134 Days
• Field of Search: 348143-160
• US Class Current: 1/1
• CPC Class Codes: G08B 13/19645 Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A video surveillance system includes a plurality of video sources, at least one processing server in communication with the video sources, and at least one monitoring station viewing video from the video sources. The video sources are distributed over a target area to be monitored by the surveillance system and are attached to a video network. The processing server allocates bandwidth to the video sources with an accuracy function which provides event/object detection accuracy as a function of image/video quality transferred over the network. The image accuracy is the accuracy with which the processing server identifies features in a video image. Bandwidth is allocated by optimizing the overall event/object detection accuracy by adjusting image/video quality for each video source optionally subject to the dynamic network conditions experienced by each video source.

Claims

1. A video surveillance system comprising:

a plurality of video sources distributed over a target area to be monitored by the video surveillance system, the video sources attached to a video network;

at least one processing server in communication with the video sources, the processing server allocating bandwidth to the video sources with an accuracy function, the accuracy function giving event or object detection accuracy as a function of image or video quality transferred over the video network, the event or object detection accuracy being an accuracy with which the processing server identifies features in an image or video, bandwidth being allocated by;

a) optimizing overall event or object detection accuracy by adjusting image or video quality for each video source subject to dynamic network conditions experienced by each video source; and

at least one monitoring station viewing video from the video sources.

2. The system of claim 1 wherein the accuracy function is a rate-accuracy function giving event or object detection accuracy as a function of the image or video bitrate transferred over the video network.

3. The system of claim 1 wherein the event or object detection accuracy is optimized by adjusting one or more video parameters selected from the group consisting of image resolution, frame rate, compression quantization and combinations thereof.

4. The system of claim 1 wherein the processing server is operable to reduce the image or video quality independently for each video source while maintaining the event or object detection accuracy to within a predetermined percent of an optimal value for each video source.

5. The system of claim 1 wherein the processing server processes and filters video streams received from the video sources.

6. The system of claim 1 wherein the video sources are selected from the group consisting of video sensors, cameras, and combinations thereof.

7. The system of claim 1 wherein the bandwidth for a particular video source is increased if a threat level associated with a video source increases.

8. The system of claim 1 wherein the processing servers and the monitoring stations are each independently computer systems having a computer processor, computer memory and data storage units.

9. The system of claim 1 wherein bandwidth allocation for each video source considers one or more factors selected from the group consisting of threat level, location of video sources, location importance, site map, dynamic network conditions, and combinations thereof.

10. The system of claim 1 wherein bandwidth allocated to each location i in the plurality of video sources is proportional to a weighting factor osw_i given by the following formula:

osw_i=z₁×

LS_i+z₂×

SS_i+z₃×

LT_i+z₄×

ST_i

LS_i is an operator assigned location-based sensitivity factor quantifying importance of location i;

SS_i is a site-based sensitivity factor quantifying importance of locations proximate to location i;

LT_i is a location-based threat sensitivity factor quantifying a detected threat level at location i;

ST_i is site-based threat sensitivity factor quantifying the detected threat level at locations proximate to location i; and

z₁, z₂, z₃, z₄ are predetermined factors quantifying a contribution of each sensitivity factors.

11. The system of claim 10 wherein

12. The system of claim 11 wherein ST_i is determined from the following formula:

13. The system of claim 1 wherein the processing server is operable to use a cross-layer optimization method to allocate bandwidth.

14. The system of claim 13 wherein a cross-layer optimization method adjusts parameters for an application layer, link layer, and physical layer.

15. The system of claim 14 wherein bandwidth allocation for each video source in the plurality of video sources is determined by finding an optimal video data rate for each video source that minimizes a total weighted event or object detection error (Σ

_s=1^Sw_s(1−

accuracy_s(r_s))), where r_s is an application layer transfer rate for video source s,

S is the number of video sources, and w_s is a weight for video source s indicating its current importance.

16. The system of claim 14 wherein bandwidth allocation for each video source in the plurality of video sources is determined by finding an optimal video data rate for each video source that minimizes a total weighted event or object detection error Σ

_s=1^Sw_s(1−

accuracy_s(vp_s)) where vp_s is video quality parameters for video source s, S is the number of video sources, and w_s is a weight for video source s indicating its current importance.

17. The system of claim 14 wherein bandwidth allocation for each video source in the plurality of video sources is determined by finding an optimal fraction of the airtime (f_s*) for each video source that minimizes a total weighted event or object detection error (Σ

_s=1^Sw_s(1−

accuracy_s(r_s))), where r_s is an application layer transfer rate for video source s, S is the number of video sources, and w_s is a weight for video source s indicating its current importance.

18. The system of claim 17 wherein determination of the optimal fraction of airtime is subject to the following constraints:

total air time of all video sources is less than an effective airtime of medium (EA);

Σ

_s=1^Sf_s≦

EA;

an application layer transfer rate of source s is a product of its airtime (f_s) and a physical layer transfer rate (y_s) for video source s;

r_s=f_s×

y_s

airtime of each source is between 0 and 1 (inclusive);

0≦

f_s≦

1;

detection accuracy is between 0 and 1 (inclusive) 0<

accuracy_s(r_s)≦

1; and

s is from 1 to S.

19. The system of claim 1 wherein the processing server is operable to select a single video source when an object is tracked by more than one video source.

20. The system of claim 1 wherein the system receives input from non-video sources.

21. The system of claim 1 wherein bandwidth is allocated proportional to a sum of sensitivity weights for threat level, placement of video sources, location importance, and site map.

22. The system of claim 9 wherein detection accuracy of each video source is weighted by a weighting factor based on an operator assigned location-based sensitivity factor quantifying importance of its location, a site-based sensitivity factor quantifying importance of locations proximate to its location, a location-based threat sensitivity factor quantifying detected threat level at its location, and a site-based threat sensitivity factor quantifying detected threat level at locations proximate to it, and combinations thereof.

23. The system of claim 15 wherein the weight for each video source w_s is set to an overall sensitivity weight osw_i of its location i.

24. The system of claim 16 wherein the weight for each video source w_s is set to an overall sensitivity weight osw_i of its location i.

25. The system of claim 17 wherein the weight for each video source w_s is set to an overall sensitivity weight osw_i of its location i.

26. The system of claim 4 wherein reduction in bitrate of each video source is related to one or more factors selected from the group consisting of threat level, remaining battery energy or lifetime of the video source, importance of a location of the video source, importance of locations leading to that location, detected threats, events, or objects in that location, detected threats, events, or objects in locations leading to that location, site map, and combinations thereof.