Content-aware computer networking devices with video analytics for reducing video storage and video communication bandwidth requirements of a video surveillance network camera system

• US 9,325,951 B2
• Filed: 12/03/2013
• Issued: 04/26/2016
• Est. Priority Date: 03/03/2008
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Avigilon Patent Holding 2 Corporation
• Inventors: Saptharishi, Mahesh
• Primary Examiner(s): Li, Tracy
• Assistant Examiner(s): Pham, Nam
• Application Number:
  US14/095,997
• Publication Number:
  US 20140085480A1
• Time in Patent Office: 875 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04N 21/23113 involving housekeeping operations for stored content, e.g. prioritizing content for deletion because of storage space restrictions storage management, e.g. defragmentation G06F3/0604; snloading stored programs G06F9/445; housekeeping operations in file systems, e.g. deletion policies G06F16/10; buffering arrangements in a network node or in an end terminal in packet networks H04L49/90
• CPC Class Codes: H04N 21/2385 Channel allocation H04N21/266 takes precedence; Bandwidth allocation H04N21/24 takes precedence ; allocation of channels according to the instantaneous demands of the users in time-division multiplex systems H04J3/1682; admission control, resource allocation in open networks H04L12/5692; arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896; negotiating bandwidth in wireless networks H04W28/16
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/64738 Monitoring network characteristics, e.g. bandwidth, congestion level data switched network analysis H04L41/14; monitoring functioning in data switched networks H04L43/0817; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/64792 Controlling the complexity of the content stream, e.g. by dropping packets intermediate media network packet handling H04L65/765; proxy provisioning conversion or adaptation for reducing the amount or size of exchanged application data H04L67/5651; negotiation of resources in wireless networks H04W28/16
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 9/7921 : for more than one processing mode
• CPC Class Codes: H04N 9/8042 : involving data reduction

Abstract

Disclosed are systems and methods for reducing video communication bandwidth requirements of a video surveillance network camera system that includes network communication paths between network video cameras producing video streams of scenes observed by the network video cameras and content-aware computer networking devices analyzing by video analytics video visual content of the video streams to provide managed video representing, at specified quality levels, samples of the scenes observed. Distribution of the managed video consumes substantially less network bandwidth than would be consumed by delivery through network communication paths a video stream at the specified quality level in the absence of analysis by the video analytics.

Claims

1. A method of reducing video communication bandwidth requirements within a video surveillance camera network including network communication paths between network video cameras and one or more network devices, the method comprising:

associating one or more of the network video cameras with the one or more network devices in one or more of the network communication paths, the one or more network devices being a router or switch, the one or more network video cameras being configured to produce video streams of scenes observed by the one or more network video cameras, the router or switch including video analytics and being configured to analyze video visual content within the video streams to produce managed video;

receiving at the router or switch the video visual content and configuration information, the configuration information defining an event of interest in the scene and configuring the switch or router to prepare the managed video in response to the video analytics detecting the event of interest in the video visual content, wherein managed video including the event of interest in the scene is prepared at a first quality level and managed video not including the event of interest in the scene is prepared as a second quality level;

storing the managed video in a local mass storage device that is contained in or connected to a housing of the switch or router; and

distributing the managed video through at least one of the network communication paths such that distribution of the managed video consumes substantially less network bandwidth than would be consumed by distribution through the at least one of the network communication paths in the absence of analysis by the video analytics.

2. The method of claim 1, wherein the managed video excludes portions of the video stream based on whether the portions include video visual content corresponding to events of interest.

3. The method of claim 1, wherein based on object characteristics and actions recognized from the video visual content, configuring the switch or router to prepare managed video includes determining whether portions of the video stream are to be discarded, contemporaneously forwarded and discarded, contemporaneously forwarded and stored in the local mass storage device, or stored in the local mass storage device and made available for subsequent distribution.

4. The method of claim 1, wherein the one or more network devices further include one or more of a console workstation, a remote mass storage device, a server, a video recorder, and a mobile communications user equipment device.

5. The method of claim 1, wherein the router operates at an Open System Interconnection (OSI) Model Layer 3 or the switch operates at OSI Model Layer 2.

6. The method of claim 1, wherein the event of interest comprises a predetermined period during which the video stream is regularly sampled and archived.

7. The method of claim 1, wherein the event of interest comprises an object of interest or an object action of interest.

8. The method of claim 1, wherein the first quality level and the second quality level of the managed video is prepared by transcoding the video stream from a first bit rate to a second bit rate different from the first bit rate.

9. The method of claim 1, wherein the associating comprises generating a routing table of nodes configured to communicate through the network communication paths.

10. The method of claim 9, wherein the routing table includes, for each of the network devices, a description of fault tolerance and load balancing resources available among the network devices.

11. The method of claim 10, wherein the fault tolerance and load balancing resources available among the network devices include video processing, video analytics, or redundant mass storage resources.

12. The method of claim 1, wherein the event of interest is characterized by a sequence of objects or actions of interest, and wherein the video visual content comprises pre-analyzed video visual content previously analyzed by one or more network devices to identify at least one object or action of interest in the sequence of objects or actions of interest.

13. The method of claim 1, wherein the managed video represents fields of view of the scenes observed, and the event of interest includes motion detected within one or more of the fields of view.

14. The method of claim 1, wherein the managed video data represent fields of view of the scenes observed, and the event of interest includes a predefined object appearing within one or more of the fields of view.

15. The method of claim 14, wherein the video analytics uses geometric calibration to recognize the predefined object as the predefined object moves to various portions within the one or more of the fields of view.

16. The method of claim 1, wherein a portion of the managed video is supplied to a remote mass storage device at a selected time characterized by low network activity of the video surveillance camera network.

17. The method of claim 1, wherein the managed video stored in the local mass storage device includes video that represents and does not represent the event of interest.

18. The method of claim 1, wherein the router or switch includes a set of video storage rules including different video retention policies for managed video representing and not representing the event of interest.

19. The method of claim 1, further comprising storing in the local mass storage device the managed video encoded at a first quality level in response to the event of interest being detected, and storing in the local mass storage device the managed video encoded at a second quality level in response to the event of interest not being detected, the first and second quality levels being different from each other.

20. The method of claim 1, wherein the configuration information includes a ranking of importance levels of multiple events of interest, the importance levels defining distribution or storage priorities for the managed video.

21. The method of claim 1, wherein the distributing includes distributing the managed video to a recipient device, in which the recipient device is a sensor, a router, a switch, a server, or a user equipment device including a user interface configured to display the managed video.

22. The method of claim 1, wherein the configuration information is received from a task compiler, the configuration information including a routing table generated by the task compiler based on user input selections of events of interest, the routing table defining, based on the events of interest, recipients of the video streams and of the managed video.

23. A content-aware computer network device for analyzing video content, comprising:

a network device positioned within a network between one or more video cameras and one or more client devices and configured to analyze and route packets of video stream data from the one or more video cameras to the one or more client devices via at least one of one or more network communication paths based on routing configuration information for the network device, the network device being configured to receive the packets of video stream data, the video stream data representing scenes observed by one or more video cameras connected to the network device over a computer network, the network device including video analytics and being further configured to analyze video content within the packets of video stream data using the video analytics, the network device being further configured to receive configuration information defining an event of interest in the scene and to configure the network device to prepare and distribute managed video in response to the video analytics detecting the event of interest in the video content, the managed video representing samples of the scene at specified quality levels.

24. A content-aware computer network device as recited in claim 23, wherein the network device is a router.

25. A content-aware computer network device as recited in claim 23, wherein the network device is a switch.