Video and audio content analysis system

• US 7,346,186 B2
• Filed: 01/28/2002
• Issued: 03/18/2008
• Est. Priority Date: 01/30/2001
• Status: Active Grant

• Current Assignee: Qognify Ltd. (PSBU Midco SARL)
• Original Assignee: Nice Systems Limited (Nice Ltd)
• Inventors: Girmonski, Doron, Katzman, Yehuda, Sharoni, David, Katz, Hagai
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US10/056,049
• Publication Number:
  US 20020110264A1
• Time in Patent Office: 2,241 Days
• Field of Search: 382/103, 382/106, 382/107, 382/115, 382/118, 382/123, 382/151, 382/156, 382/169, 382/174, 382181-184, 382/192, 382/199, 382/203, 382/206, 382/233, 382/274, 382/285, 382/291, 382/295, 382/305, 382/232, 382/254, 340/572.4, 340/506, 340/572.3, 348/143, 348/14.12, 348/14.2, 370/352, 370/384, 379/45
• US Class Current: 382/103
• CPC Class Codes: G08B 13/19645 Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over
• CPC Class Codes: G08B 13/19697 Arrangements wherein non-video detectors generate an alarm themselves

Abstract

The present invention is directed to various methods and systems for analysis and processing of video and audio signals from a plurality of sources in real-time or off-line. According to some embodiments of the present invention, analysis and processing applications are dynamically installed ill the processing units.

Claims

1. A system comprising:

two or more processing units, each coupled to a respective video or audio sensor, each of which receives video or audio data from said sensor and processes said video or audio data according to one or more content-analysis applications installed therein;

an application bank external to said processing units, said application bank comprising said content-analysis applications; and

a control unit having installed therein a set of post-alert action rules, the control unit coupled to said processing units and to said application bank, said control unit automatically, without user input, and dynamically instructs said application bank to install in real-time, alter receiving an alert from one or more of said processing units, at least another one of said content-analysis applications into at least one of said processing units based on at least one of said post-alert action rules;

wherein said processing unit is able to notify said control unit when one of said applications installed in said processing unit detects a predefined condition associated with at least a portion of said audio or video data;

wherein said control unit is able to instruct one of said processing units to activate or deactivate one of said content-analysis applications already installed in said processing unit based on an alert received from said processing unit or another one of said processing units; and

providing to a client computer a real-time stream of video data, audio data or a combination thereof according to a predetermined time-based schedule.

2. The system of claim 1, wherein at least one of said content-analysis applications is a video movement-detecting application, a video based people counting application, a face detection and recognition application, a voice detection and recognition application, an object detection application or a recognition and surveillance application.

3. The system. of claim 1, wherein said application bank further comprises at least a conversion of speech to text application or a video compression application.

4. The system of claim 1 further comprising at least one additional processing unit coupled to a sensor, which is a smoke sensor, a fire sensor, a motion detector, a sound detector, a presence sensor, a movement sensor, a volume sensor or a glass breakage sensor.

5. The system of claim 1 further comprising a database to store indexing data associated with said video or audio data.

6. The system of claim 1, wherein said application bank, said control unit and said processing units are all coupled via a local area or a wide area network.

7. The method of claim 1 wherein providing said real-time data comprises providing synchronized video data received from at least two sensors.

8. The system of claim 1, wherein one of said post-alert action rules is to install a face recognition application upon detection of a movement.

9. The system of claim 1, Wherein one of said post-alert action rules is to install a video compression application upon identification of a particular person.

10. The system of claim 1, wherein one of said post-alert action rules is to install a face recognition application upon identification of the voice of a particular person.

11. A system comprising:

an application bank having one or more content-analysis applications;

a processing unit coupled to said application, bank and to a video sensor, the application bank being external to said processing unit, wherein said processing unit receives video data from said sensor, processes said data according at least one of said content-analysis applications installed therein, and sends an alert when a predefined condition associated with at least a portion of said data is detected; and

a control unit having installed therein a set of post-alert action rules, the control unit coupled to said processing unit and to said application bank, said control unit is to instruct automatically, without user input, and dynamically said application bank to install, in real-time after receiving the alert, another one of said content-analysis applications into said processing unit according to at least one of said post-alert action rules;

wherein said processing unit is able to notify said control unit when one of said applications installed in said processing unit detects a predefined condition associated with at least a portion of said audio or video data;

wherein said control unit is able to instruct one of said processing units to activate or deactivate one of said content-analysis applications already installed in said processing unit based on an alert received from said processing unit or another one of said processing units;

providing to a client computer a real-time stream of video data, audio data or a combination thereof according to a predetermined time-based schedule.

12. A method comprising:

storing predefined post-alert action rules in a control unit;

detecting a predefined condition associated with at least a portion of an audio or video data received from a video or audio sensor by processing the data at a processing unit coupled to the sensor according to one or more content-analysis applications;

sending an alert based on the detected predefined condition; and

automatically, without user input, and dynamically in real-time after receiving the alert, instructing to install another content-analysis application into a video or audio processing unit from an application bank external to said processing unit having content-analysis applications based on at least one of said predefined post-alert action rules;

wherein said processing unit is able to notify said control unit when one of said applications installed in said processing unit detects a predefined condition associated with at least a portion of said audio or video data;

wherein said control unit is able to instruct one of said processing units to activate or deactivate one of said content-analysis applications already installed in said processing unit based on an alert received from said processing unit or another one of said processing units;

providing to a client computer a real-time stream of video data, audio data or a combination thereof according to a predetermined time-based schedule.

13. The method of claim 12 further comprising:

recording at least a portion of said data.

14. The method of claim 13 further comprising:

providing to a client computer recorded data upon receiving a request from said client computer.

15. The method of claim 13 further comprising:

down-loading at feast one content-analysis application from said application bank to a client computer;

providing to said client computer recorded data upon receiving a request from said client computer; and

processing said recorded data according to at least one of said installed applications.

16. The method of claim 15 wherein providing said recorded data comprises providing synchronized video data received from at least two sensors.

17. The method of claim 13 further comprising:

storing results of said content-analysis applications.

18. The method of claim 17, wherein the results comprises number of people counted by a people counting application.

19. The method of claim 17, wherein the results comprises an identifier associated with a voice recognized by a voice recognition application.

20. The method of claim 17, wherein the results comprises an identifier associated with a face recognized by a face recognition application.

21. The method of claim 12 further comprising:

providing to a client computer a real-time stream of video data, audio data or a combination thereof upon receiving a request from said client computer.

22. The method of claim 21 wherein providing said real-time data comprises providing synchronized video data received from at least two sensors.

23. A method comprising:

installing one or more content-analysis applications from an application bank into one or more video or audio processing units, the application bank being external to said processing units;

storing predefined post-alert action rules in a control unit;

processing input received from one or more video or audio sensors, each coupled to a respective video or audio processing unit according to at least one of said content-analysis applications;

detecting a predefined condition associated with at least one portion of said input;

sending a notification associated with said condition to a control unit; and

automatically, without user input, and dynamically in real-time after receiving the notification, instructing the application bank to install at least another, one of said content-analysis applications into at least one of said processing units based on at least one of said pro defined post-alert action rules;

wherein said processing unit is able to notify said control unit when one of said applications installed in said processing unit detects a predefined condition associated with at least a portion of said audio or video data;

wherein said control unit is able to instruct one of said processing units to activate or deactivate one of said content-analysis applications already installed in said processing unit based on an alert received from said processing unit or another one of said processing units;

providing to a client computer a real-time stream of video data, audio data or a combination thereof according to a predetermined time-based schedule.