NETWORK BASED VIDEO ANALYTICS THROUGH AN APPLICATION PROGRAM INTERFACE (API) WITH METRIC TRIGGERED NOTIFICATIONS

• US 20150062338A1
• Filed: 11/13/2014
• Published: 03/05/2015
• Est. Priority Date: 07/25/2008
• Status: Active Grant

• Current Assignee: AirBNB Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: MARIADOSS, PANDIAN
• Granted Patent: US 9,277,250 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19669 Event triggers storage or change of storage policy
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/443 OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
• CPC Class Codes: H04N 21/4431 characterized by the use of Application Program Interface [API] libraries
• CPC Class Codes: H04N 21/478 Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/4882 for displaying messages, e.g. warnings, reminders arrangements for providing short real-time information to substation in data switching networks H04L12/1895
• CPC Class Codes: H04N 21/6175 involving transmission via Internet broadcast-related systems characterised by the transmission system being the Internet H04H60/82
• CPC Class Codes: H04N 21/64322 IP
• CPC Class Codes: H04N 21/814 comprising emergency warnings arrangements specially adapted for emergency or urgency in broadcast systems H04H20/59; arrangements for providing alarms, notifications, alerts to substation in data switching networks H04L12/1895
• CPC Class Codes: H04N 21/8352 : involving content or source identification data, e.g. Unique Material Identifier [UMID]
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources

Abstract

A video surveillance system ingests a video stream from a remotely located video capture device. An application programming interface (API) provides an interface for functions to interface between the video surveillance system and the remotely located video capture device. An analytics engine of the video surveillance system receives video sent through the API. The analytics engine compares an extracted metric from the video stream against at least one previously established value as part of assessing a previously established condition to generate either a TRUE or a FALSE result. When and only when a TRUE result is generated, is a notification action performed that conveys a notification over a network using internet protocol communications to a previously designated user.

Claims

1. A system for performing video analytics comprising:

a video surveillance system, comprising one or more processors for ingesting a video stream from a remotely located video capture device, wherein said video surveillance system performs real-time or near-real time video analytics on the video stream;

an application programming interface (API) for providing an interface for functions to interface between the video surveillance system and the remotely located video capture device; and

a software engine, comprising program instructions stored in at least one non-transitory medium, said program instructions able to be executed by at least one processor to perform at least one programmatic action responsive to results produced by an analytics engine that performs the real-time analytics, wherein the analytics engine;

compares an extracted metric from the video stream against at least one previously established value as part of assessing a previously established condition to generate either a TRUE or a FALSE result;

responsive to a TRUE result, automatically performs a notification action that conveys a notification to a previously designated user when the previously established condition is automatically detected by the analytics engine,

responsive to a FALSE comparison result, not performing the notification action.

2. The system of claim 1, wherein different filters specific to different remotely located video capture devices are used by the video surveillance system, wherein a specific one of the different filters must be used that corresponds to a specific type of remotely located video capture devices else the video surveillance system is not able to process the video stream.

3. The system of claim 1, wherein the remotely located video capture device is a network connected Internet Protocol (IP) camera IP, wherein the API is configured to decode the video stream from the network connected IP camber before conveying the decoded video stream to the analytics engine.

4. The system of claim 1, wherein the extracted metric is a metric specific to a human face, which is a result of a facial recognition action performed by the analytics engine.

5. The system of claim 1, wherein the extracted metric is a metric specific to a tracked object, which is a result of an object tracking action performed by the analytics engine.

6. The system of claim 1, wherein the extracted metric is a metric specific to a movement within a scene as determined by comparing a previous image of the scene from a current image and ascertaining a delta representative of a significant motion occurring by an object in the scent, wherein the analytics engine determines the movement within the scene via a motion detection operation.

7. The system of claim 1, wherein internet connected computers are able to send authorized commands to the video surveillance system directed to the remotely located video capture device, wherein the video surveillance system conveys signals through the API to the remotely located video capture device to effectuate an adjustment of the remotely located video capture device to change an aspect of the video stream captured by the remotely located video capture device.

8. A method comprising:

one or more computers receiving a video stream over a network from a remotely located video capture device connected over an internet protocol communication channel via an application program interface (API);

at least one of the one or more computers decoding the video stream to produce a decoded video stream;

at least one of the one or more computers processing the decoded video stream, wherein the processing of the video stream extracts at least one metric from content of the decoded video stream;

at least one of the one or more computers comparing the extracted metric against at least one previously established value to generate either a TRUE or a FALSE result;

responsive to a TRUE result, at least one of the one or more computers performing a notification action that conveys a notification to a previously designated user when the previously established condition is automatically detected by the analytics engine; and

responsive to a FALSE comparison result, none of the one or more computers performing the notification action.

9. The method claim 8, further comprising:

at least one of the one or more computers selecting at least one of a plurality of different video decoding filters, wherein the selected filter corresponds to specifics of the remotely located video capture device, wherein other types of video capture devices correspond to other ones of the different video decoding filters;

performing the decoding of the video stream using the selected filter to produce the decoded video stream;

10. The method of claim 8, wherein the video stream captures input from a geospatial region, wherein an authorized entity is able to configure the established value, the programmatic action, or both is has responsibility and authority over the geospatial region.

11. The method of claim 8, wherein the video capture device is a network connected IP camera.

12. The method of claim 8, wherein the received video stream is raw, unprocessed video captured by a video capture device.

13. The method of claim 8, wherein the processing of the decoded video stream and the comparing of the extracted metric are performed as part of a face recognition analysis of the video stream performed by the one or more computers, wherein the notification indicates an identity of a human that results from the face recognition analysis.

14. The method of claim 8, wherein the processing of the decoded video stream and the comparing of the extracted metric are performed as part of a motion detection analysis of the video stream performed by the one or more computers.

15. The method of claim 8, wherein the processing of the decoded video stream comprises:

at least one of the one or more computers exporting metadata from the video stream; and

at least one of the one or more computers indexing the metadata within a database system.

16. A computer program product comprising:

a non-transitory computer usable storage medium storing computer usable program code;

computer usable program code being executable by at least one processor to;

receive a video stream over a network from a remotely located video capture device connected over an internet protocol communication channel via an application program interface (API);

select at least one of a plurality of different video decoding filters, wherein the selected filter corresponds to specifics of the remotely located video capture device, wherein other types of video capture devices correspond to other ones of the different video decoding filters;

decode the video stream using the selected filter to produce a decoded video stream;

process the decoded video stream, wherein the processing of the video stream extracts at least one metric from content of the decoded video stream;

compare the extracted metric against at least one previously established value to generate either a TRUE or a FALSE result;

responsive to a TRUE result, perform a notification action that conveys a notification to a previously designated user when the previously established condition is automatically detected by the analytics engine; and

responsive to a FALSE comparison result, not performing the notification action.

17. The computer program product of claim 16, wherein the video stream captures input from a geospatial region, wherein an authorized entity is able to configure the established value, the programmatic action, or both is has responsibility and authority over the geospatial region.

18. The computer program product of claim 16, wherein the video capture device is a network connected IP camera.

19. The computer program product of claim 16, wherein processing of the decoded video stream and comparing of the extracted metric are performed as part of a face recognition analysis of the video stream, wherein the notification indicates an identity of a human that results from the face recognition analysis.

20. The computer program product of claim 16, the computer usable program code being executable by at least one processor to further send authorized commands to the video surveillance system directed to the remotely located video capture device, wherein the video surveillance system conveys signals through the API to the remotely located video capture device to effectuate an adjustment of the remotely located video capture device to change an aspect of the video stream captured by the remotely located video capture device.