Bandwidth efficient video surveillance system

• US 10,616,465 B2
• Filed: 09/16/2015
• Issued: 04/07/2020
• Est. Priority Date: 09/16/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Chowdhery, Aakanksha, Bahl, Paramvir, Zhang, Tan
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): Walsh, Kathleen M
• Application Number:
  US14/856,340
• Publication Number:
  US 20170078626A1
• Time in Patent Office: 1,665 Days
• Field of Search: 348143-159
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A system and method are disclosed for providing a real-time wireless video surveillance system. The video surveillance system leverages edge computing to enable wireless video surveillance distributing video processing between edges of the network and the cloud to reduce the amount of video that is uploaded to the cloud for analysis.

Claims

1. A surveillance system, comprising:

a plurality of computing devices, associated with a plurality of video cameras capturing videos of a geographic area, the plurality of computing devices configured to process the videos to generate analytic data, the analytic data from the plurality of computing devices indicating utility detected in the videos during a period of time relative to search criteria received for the videos, wherein utility is a measure of a number of tracked items detected in the videos relevant to the search criteria received for the videos;

a controller configured to receive the analytic data from the plurality of computing devices relating to the search criteria for the videos, and to select a video to upload from a computing device of the plurality of computing devices during the period of time based on the utility of the video to the search criteria for the videos, the controller requesting the video from the computing device having a highest average utility of tracked items to the search criteria for the videos over the period of time to upload; and

a wireless transmitter for wirelessly transmitting the analytic data from the computing device containing the video selected by the controller,

wherein the controller estimates a wireless transmitter transmission capacity by examining near-term sending rates and loss rates of the wireless transmitter to estimate a number of bits that the wireless transmitter can upload within the period of time and the wireless transmitter transmits all of the video during the period of time when a product of wireless transmit capacity of the wireless transmitter and a length of the period of time is greater than a size of the video frames to be uploaded in the period of time and transmits part of the video during the period of time, in a decreasing order of utility, when the product of the wireless transmit capacity of the wireless transmitter and a length of the period of time is less than a size of the video frames to be uploaded during the period of time.

2. The surveillance system of claim 1, wherein the analytic data for the computing device includes a frame index and a frame utility of tracked items in a period of time associated with the frame index.

3. The surveillance system of claim 2, wherein the frame utility of tracked items associated with the frame index is different than a frame utility of tracked items in a frame index associated with a next-earlier period of time.

4. The surveillance system of claim 1, wherein the analytic data for the computing device includes one or more frame indices representing one or more periods of time having a different number of tracked items than one or more immediately-preceding periods of time, and a count of tracked items in the one or more periods of time associated with the one or more frame indices.

5. The surveillance system of claim 1, wherein utility for the plurality of computing devices is a measure of one of:

i) numbers of tracked items in periods of time, ii) whether a tracked item is detected or not detected in the periods of time, and iii) a probability measure of whether a tracked item is detected in the periods of time.

6. The surveillance system of claim 1, wherein a computing device of the plurality of computing devices is co-located with a video camera of the plurality of video cameras.

7. The surveillance system of claim 1, wherein a computing device of the plurality of computing devices is connected with a video camera of the plurality of video cameras by a wired connection.

8. A surveillance system, comprising:

a plurality of computing devices, associated with a plurality of video cameras capturing videos of items in a geographic area, the plurality of computing devices configured to process the videos to generate analytic data, the analytic data from the plurality of computing devices indicating utility of video detected in the videos during a period of time relative to search criteria received for the videos, wherein utility is a measure of a number of tracked items detected in the videos relevant to the search criteria received for the videos;

a wireless network; and

a controller that communicates with the plurality of computing devices at least in part via the wireless network, the controller selecting a plurality of video frames for upload from one or more of the plurality of computing devices via the wireless network based on the analytic data received from the plurality of computing devices relating to the search criteria for the videos, the controller prioritizing an order in which the plurality of video frames are uploaded based on bandwidth availability of the wireless network and the analytic data of the video frames to be uploaded during the period of time, the analytic data identifying a computing device having a highest average utility of tracked items to upload during the period of time,

wherein the wireless network allocates bandwidth for use as a wi-fi hotspot during the period of time when less than an entire bandwidth of the wireless network is taken up transmitting image data from the one or more computing devices.

9. The surveillance system of claim 8, wherein the entire plurality of video frames are uploaded if there is available bandwidth at a time when the entire plurality of video frames are sent.

10. The surveillance system of claim 8, wherein a portion of the plurality of video frames is uploaded if there is not available bandwidth to send the entire plurality of video frames at a time when the entire plurality of video frames are sent, the portion comprising one or more video frames from a computing device having a highest average utility over the period of time.

11. The surveillance system of claim 10, wherein the portion of the plurality of video frames from the computing device further comprise the video frame from the computing device having a highest utility over the period of time.