Content-aware computer networking devices with video analytics for reducing video storage and video communication bandwidth requirements of a video surveillance network camera system

• US 9,756,294 B2
• Filed: 03/22/2016
• Issued: 09/05/2017
• Est. Priority Date: 03/03/2008
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Avigilon Analytics Corporation
• Inventors: Saptharishi, Mahesh
• Primary Examiner(s): PHAM, NAM D
• Application Number:
  US15/077,822
• Publication Number:
  US 20160205422A1
• Time in Patent Office: 532 Days
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04N 21/23113 involving housekeeping operations for stored content, e.g. prioritizing content for deletion because of storage space restrictions storage management, e.g. defragmentation G06F3/0604; snloading stored programs G06F9/445; housekeeping operations in file systems, e.g. deletion policies G06F16/10; buffering arrangements in a network node or in an end terminal in packet networks H04L49/90
• CPC Class Codes: H04N 21/2385 Channel allocation H04N21/266 takes precedence; Bandwidth allocation H04N21/24 takes precedence ; allocation of channels according to the instantaneous demands of the users in time-division multiplex systems H04J3/1682; admission control, resource allocation in open networks H04L12/5692; arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896; negotiating bandwidth in wireless networks H04W28/16
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/64738 Monitoring network characteristics, e.g. bandwidth, congestion level data switched network analysis H04L41/14; monitoring functioning in data switched networks H04L43/0817; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/64792 Controlling the complexity of the content stream, e.g. by dropping packets intermediate media network packet handling H04L65/765; proxy provisioning conversion or adaptation for reducing the amount or size of exchanged application data H04L67/5651; negotiation of resources in wireless networks H04W28/16
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 9/7921 : for more than one processing mode
• CPC Class Codes: H04N 9/8042 : involving data reduction

Abstract

Disclosed are systems and methods for reducing video communication bandwidth requirements of a video surveillance network camera system that includes network communication paths between network video cameras producing video streams of scenes observed by the network video cameras and content-aware computer networking devices analyzing by video analytics video visual content of the video streams to provide managed video representing, at specified quality levels, samples of the scenes observed. Distribution of the managed video consumes substantially less network bandwidth than would be consumed by delivery through network communication paths a video stream at the specified quality level in the absence of analysis by the video analytics.

Claims

1. A method of reducing video communication bandwidth requirements within a video surveillance camera network, the method comprising:

associating one or more video cameras with one or more network devices positioned within the network, the network including one or more network communication paths between the one or more video cameras and one or more client devices, the one or more video cameras being configured to produce packets of video stream information based on scenes observed by the one or more video cameras, the one or more network devices including video analytics and being further configured to analyze video content within the video stream information using the video analytics;

receiving at the one or more network devices the packets of video stream information and routing configuration information;

analyzing and routing the packets of video stream information from the one or more video cameras to the one or more client devices via at least one of one or more network communication paths based on the routing configuration information, the routing configuration information defining an event of interest in a scene and configuring the one or more network devices to prepare managed video in response to the video analytics detecting the event of interest in the video content, wherein managed video including the event of interest in the scene is prepared at a first quality level and managed video not including the event of interest in the scene is prepared as a second quality level;

storing the managed video in a local mass storage device that is contained in or connected to a housing of the one or more network devices; and

distributing the managed video through at least one of the one or more network communication paths such that distribution of the managed video consumes substantially less network bandwidth than would be consumed by distribution through the at least one of the one or more network communication paths in the absence of analysis by the video analytics.

2. The method of claim 1, wherein the managed video excludes portions of the video content based on whether the portions include the video content corresponding to events of interest.

3. The method of claim 1, wherein based on object characteristics and actions recognized from the video content, configuring the one or more network devices to prepare managed video includes determining whether portions of the video content are to be discarded, contemporaneously forwarded and discarded, contemporaneously forwarded and stored in the local mass storage device, or stored in the local mass storage device and made available for subsequent distribution.

4. The method of claim 1, wherein the one or more client devices further include one or more of a console workstation, a remote mass storage device, a server, a video recorder, and a mobile communications user equipment device.

5. The method of claim 1, wherein the one or more network devices operate at an Open System Interconnection (OSI) Model Layer 3 or OSI Model Layer 2.

6. The method of claim 1, wherein the event of interest comprises a predetermined period during which the video content is regularly sampled and archived.

7. The method of claim 1, wherein the event of interest comprises an object of interest or an object action of interest.

8. The method of claim 1, wherein the first quality level and the second quality level of the managed video is prepared by transcoding the video stream from a first bit rate to a second bit rate different from the first bit rate.

9. The method of claim 1, wherein the associating comprises generating a routing table of nodes configured to communicate through the one or more network communication paths.

10. The method of claim 9, wherein the routing table includes, for each of the one or more network devices, a description of fault tolerance and load balancing resources available among the one or more network devices.

11. The method of claim 10, wherein the fault tolerance and load balancing resources available among the one or more network devices include video processing, video analytics, or redundant mass storage resources.

12. The method of claim 1, wherein the event of interest is characterized by a sequence of objects or actions of interest, and wherein the video content comprises pre-analyzed video content previously analyzed by the one or more network devices to identify at least one object or action of interest in the sequence of objects or actions of interest.

13. The method of claim 1, wherein the managed video represents fields of view of the scenes observed, and the event of interest includes motion detected within one or more of the fields of view.

14. The method of claim 1, wherein the managed video represents fields of view of the scenes observed, and the event of interest includes a predefined object appearing within one or more of the fields of view.

15. The method of claim 14, wherein the video analytics uses geometric calibration to recognize the predefined object as the predefined object moves to various portions within the one or more of the fields of view.

16. The method of claim 1, wherein a portion of the managed video is supplied to a remote mass storage device at a selected time characterized by low network activity of the network.

17. The method of claim 1, wherein the managed video stored in the local mass storage device includes video content that represents and does not represent the event of interest.

18. The method of claim 1, wherein the one or more network devices includes a set of video storage rules including different video retention policies for managed video representing and not representing the event of interest.

19. The method of claim 1, further comprising storing in the local mass storage device the managed video encoded at a first quality level in response to the event of interest not being detected, and storing in the local mass storage device the managed video encoded at a second quality level in response to the event of interest not being detected, the first and second quality levels being different from each other.

20. The method of claim 1, wherein the routing configuration information includes a ranking of importance levels of multiple events of interest, the importance levels defining distribution or storage priorities for the managed video.

21. The method of claim 1, wherein the distributing includes distributing the managed video to a recipient device, in which the recipient device is a sensor, a router, a switch, a server, or a user equipment device including a user interface configured to display the managed video.

22. The method of claim 1, wherein the routing configuration information is received from a task compiler, the routing configuration information including a routing table generated by the task compiler based on user input selections of events of interest, the routing table defining, based on the events of interest, recipients of the video stream information and of the managed video.