System and method for providing surveillance data

• US 10,282,620 B2
• Filed: 01/05/2017
• Issued: 05/07/2019
• Est. Priority Date: 07/21/2016
• Status: Active Grant

• Current Assignee: Hanwha Vision Company Limited (Hanwha Corporation)
• Original Assignee: Hanwha Aerospace Co., Ltd. (Hanwha Corporation)
• Inventors: Noh, Seung In, Lim, Jeong Eun, Jeon, Seoung Seon
• Primary Examiner(s): Thomas, Mia M
• Application Number:
  US15/398,843
• Publication Number:
  US 20180025231A1
• Time in Patent Office: 852 Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06V 10/431 Frequency domain transformation; Autocorrelation
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/48 Matching video sequences
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A system and method for providing surveillance data are provided. The system includes: a pattern learner configured to learn a time-based data pattern by analyzing at least one of image data of one or more images and sound data of sound obtained from a surveillance zone at a predetermined time or time period, and to generate an event model based on the time-based data pattern; and an event detector configured to detect at least one event by comparing the event model with a time-based data pattern of at least one of first image data of one or more first images and first sound data of first sound obtained from the surveillance zone.

Claims

1. A system for providing surveillance data comprising:

a pattern learner, implemented by at least one processor, configured to learn a first time-based data pattern by analyzing at least one of image data of one or more images and sound data of sound obtained from a surveillance zone at a predetermined time or time period, and to generate an event model based on the first time-based data pattern; and

an event detector, implemented by the at least one processor, configured to detect at least one event by comparing the event model with a second time-based data pattern of at least one of first image data of one or more first images and first sound data of first sound obtained from the surveillance zone,

wherein the first time-based data pattern represents a time-based variation in the at least one of the image data and the sound data, and the event detector is configured to detect the at least one event in response to determining that at least one of the first image data and the first sound data deviates, out of a preset error range, from the time-based variation of the first time-based data pattern.

2. The system of claim 1, wherein the pattern learner comprises:

a first learner configured to calculate a statistical data value of at least one of a color, a number of at least one object detected in the surveillance zone, and a degree of movement of the detected at least one object from the one or more images; and

a second learner configured to calculate a statistical data value of at least one sound level from the sound,

wherein the first time-based data pattern corresponds to a time-based variation in the statistical data value calculated by the first learner or the second learner.

3. The system of claim 2, wherein the pattern learner is configured to generate the event model based on the image data and the sound data corresponding to a first average value, of the at least one of the color, the number of the at least one object, and the degree of movement, and a second average value of the at least one sound level, calculated by the first learner and the second learner, respectively, and

wherein, in response to a value of the first image data or a value of the first sound data being identical to or greater than the first average value or the second average value, respectively, the event detector is configured to determine that the at least one event has occurred.

4. The system of claim 1, wherein the event detector is configured to detect the at least one event from the first image data and the first sound data by considering daily data variation patterns of the image data and the sound data.

5. The system of claim 4, wherein in response to at least one of the first image data and the first sound data not corresponding to the daily data variation patterns, the event detector is configured to determine that the at least one event has occurred.

6. The system of claim 1, wherein the pattern learner is configured to determine a moving path of an object, detected in the surveillance zone, based on image data obtained from a plurality of cameras photographing the surveillance zone, and

wherein the pattern learner is configured to generate the event model additionally based on a third time-based data pattern of the moving path.

7. The system of claim 1, wherein the event detector is further configured to assign priorities to the detected at least one event comprising a plurality of events, and

wherein the system further comprises an event output interface configured to generate an event list of the plurality of events according to the assigned priorities.

8. The system of claim 7, wherein the event detector is further configured to assign the priorities, different from one another, to the plurality of events based on an amount of difference between the event model and the second time-based data pattern corresponding to each of the plurality of events.

9. The system of claim 1, wherein the system further comprises or is connected to a database storing the first image data and the first sound data, and

wherein the pattern learner is configured to receive a user'"'"'s search request and update the event model according to the user'"'"'s search request.

10. The system of claim 9, wherein the event detector is further configured to assign priorities to the detected at least one event comprising a plurality of events,

wherein the system further comprises an event output interface configured to generate an event list of the plurality of events, and

wherein the event output interface is configured to search for data corresponding to the user'"'"'s search request in the database and add the data to the event list.

11. A method of providing surveillance data, the method comprising:

learning a first time-based data pattern by analyzing at least one of image data of one or more images and sound data of sound obtained from a surveillance zone at a predetermined time or time period, and to generate an event model based on the first time-based data pattern; and

detecting at least one event by comparing the event model with a second time-based data pattern of at least one of first image data of one or more first images and first sound data of first sound obtained from the surveillance zone,

wherein the first time-based data pattern represents a time-based variation in the at least one of the image data and the sound data, and the detecting the at least one event comprises detecting the at least one event in response to determining that at least one of the first image data and the first sound data deviates, out of a preset error range, from the time-based variation of the first time-based data pattern.

12. The method of claim 11, wherein the learning the first time-based data pattern comprises:

performing a first learning operation to calculate a statistical data value of at least one of a color, a number of at least one object detected in the surveillance zone, and a degree of movement of the detected at least one object from the one or more images; and

performing a second learning operation to calculate a statistical data value of at least one sound level from the sound,

wherein the first time-based data pattern corresponds to a time-based variation in the statistical data value calculated in the first learning operation or the second learning operation.

13. The method of claim 12, wherein in the learning of the first time-based data pattern, the event model is generated based on the image data and the sound data corresponding to a first average value, of the at least one of the color, the number of the at least one object, and the degree of movement, and a second average value of the at least one sound level, calculated through the first learning operation and the second learning operation, and

wherein in the detecting the at least one event, it is determined that the at least one event has occurred, in response to a value of the first image data or a value of the first sound data being identical to or greater than the first average value or the second average value, respectively.

14. The method of claim 11, wherein in the detecting the at least one event, the at least one event is detected from the first image data and the first sound data by considering daily data variation patterns of the image data and the sound data.

15. The method of claim 14, wherein the first image data and the first sound data not corresponding to the daily data variation patterns, it is determined that the at least one event has occurred.

16. The method of claim 11, further comprising determining a moving path of an object, detected in the surveillance zone, based on image data obtained from a plurality of cameras photographing the surveillance zone,

wherein the event model is generated additionally based on a third time-based data pattern of the moving path.

17. The method of claim 11, further comprising:

assigning priorities to the detected at least one event comprising a plurality of events; and

generating an event list of the plurality of events.

18. The method of claim 17, wherein the assigning the priorities comprises assigning the priorities, different from one another, to the plurality of events based on an amount of difference between the event model and the second time-based data pattern corresponding to each of the plurality of events.

19. The method of claim 11, further comprising:

storing the first image data and the first sound data in a database; and

receiving a user'"'"'s search request and updating the event model according to the user'"'"'s search request.

20. The method of claim 19, further comprising:

assigning priorities to the detected at least one event comprising a plurality of events;

generating an event list of the plurality of events; and

searching for data corresponding to the user'"'"'s search request in the database and adding the data to the event list.

21. A system for providing surveillance data comprising:

a pattern learner, implemented by at least one processor, configured to learn a first time-based data pattern by analyzing at least one of image data of one or more images and sound data of sound obtained from a surveillance zone at a predetermined time or time period, and to generate an event model based on the first time-based data pattern; and

an event detector, implemented by the at least one processor, configured to detect at least one event by comparing the event model with a second time-based data pattern of at least one of first image data of one or more first images and first sound data of first sound obtained from the surveillance zone,

wherein the pattern learner is configured to obtain a statistical data value with respect to at least one of at least one object detected from the one or more images and at least one sound level from the sound, and the first time-based data pattern corresponds to a time-based variation in the statistical data value.