Apparatus and method for automatically detecting an event in sensor data

• US 9,538,146 B2
• Filed: 10/23/2012
• Issued: 01/03/2017
• Est. Priority Date: 12/07/2011
• Status: Expired due to Fees

• Current Assignee: Siemens AG
• Original Assignee: Siemens AG
• Inventors: Buschek, Daniel, Riegel, Thomas
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US14/363,833
• Publication Number:
  US 20140347480A1
• Time in Patent Office: 1,533 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An apparatus automatically detects an event occurring in sensor data. The apparatus has a recording device which is configured to receive the sensor data, a feature identification device which is configured to automatically identify a predetermined number of features of the sensor data in the recorded sensor data, an evaluation processing device which is configured to acquire from the predetermined number of features, for each of the features, an evaluation which relates to the event to be detected and which is based on a set of evaluation criteria, and a detection device which is configured to automatically acquire the event to be detected by the features identified by the feature identification device, based on the detected evaluations for the predetermined number of features.

Claims

1. An apparatus for automatically detecting an event in sensor data, the apparatus comprising:

a recording device for recording the sensor data;

a feature identification device for automatically identifying a prescribed set of features of the sensor data in the sensor data recorded;

a rating acquisition device for acquiring a respective rating for each of the features from the prescribed set of features, the respective rating relating to the event to be detected and is formed on a basis of a set of rating criteria, the set of rating criteria being based on at least one input interactively provided by a human expert for rating each of the features;

a detection device for automatically detecting the event to be detected by means of the features identified by said feature identification device on a basis of acquired ratings for the prescribed set of features, said detection device determining a probability value for each of a plurality of individual sections of the sensor data from the acquired ratings of the features;

a display device for showing the human expert a profile over time of the probability value of the individual sections of the sensor data; and

a threshold value acquisition device for acquiring, for at least one of the features from the set of features, a threshold value formed on a basis of the set of rating criteria and to exclude from a detection of the feature that is to be detected those excerpts from the sensor data recorded for which the at least one feature is below a prescribed threshold value.

2. The apparatus according to claim 1, wherein said recording device has at least one device selected from the group consisting of cameras, microphones, position sensors, location sensors, accelerometers and brightness sensors for recording the sensor data.

3. The apparatus according to claim 1, wherein:

said feature identification device is configured to, during identification;

identify a pattern in the sensor data recorded; and

/or

identify objects contained in the sensor data that are present as an image recording and/or video recording; and

/or

identify a number of existent persons in the image recording and/or the video recording; and

/or

identify in the image recording and/or the video recording an orientation and/or position and/or direction of movement and/or movement of at least one person contained in the image recording and/or the video recording; and

/or

identify a change between at least two scenes in the video recording; and

/or

identify a sound that potentially indicates occurrence of the event to be detected in the sensor data that are present as a sound recording; and

/or

identify a change between at least two scenes in the sound recording; and

/or

identify in the sound recording at least one number of human voices contained in the sound recording.

4. The apparatus according to claim 1, wherein:

the set of rating criteria has an acceptance value and/or a rejection value and/or an uncertainty value and/or a base value; and

said rating acquisition device configured to acquire the acceptance value and/or the rejection value and/or the uncertainty value and/or the base value for each said rating.

5. The apparatus according to claim 1, wherein said detection device is configured to calculate, for extracted features, on a basis of arithmetic rules corresponding to the set of rating criteria, a respective opinion value for at least one excerpt from the sensor data recorded for the event that is to be detected.

6. The apparatus according to claim 1, wherein said detection device calculates, for the acquired ratings, on a basis of arithmetic rules corresponding to the set of rating criteria, a respective opinion value for at least one of the excerpts from the sensor data recorded for the event that is to be detected.

7. The apparatus according to claim 1, wherein said detection device, during detection, identifies at least one of the excerpts from the sensor data for which a probability of occurrence of the event that is to be detected exceeds the probability value calculated from the acquired ratings, on a basis of arithmetic rules corresponding to the set of rating criteria and opinion values calculated for the extracted features.

8. A method for automatically detecting an event in sensor data, which comprises the steps of:

recording of the sensor data;

automatically identifying a prescribed set of features of the sensor data from recorded sensor data;

modeling a respective rating for each of the features from the prescribed set of features, the respective rating relating to the event to be detected and is formed on a basis of a set of rating criteria, the set of rating criteria being based on at least one input interactively provided by a human expert for rating each of the features;

acquiring a threshold value formed on a basis of the set of rating criteria for at least one of the features from the set of features, and excerpts from the recorded sensor data for which the at least one feature is below a prescribed threshold value are excluded from a detection of the feature that is to be detected;

determining a probability value for each of a plurality of individual sections of the sensor data from acquired ratings of the features and showing the human expert a profile over time of the probability value of the individual sections of the sensor data; and

automatically detecting the event to be detected by means of identified features on a basis of the acquired ratings for the prescribed set of features.

9. The method according to claim 8, which further comprises recording the sensor data by means of at least one recording device selected from the group consisting of cameras, microphones, position sensors, location sensors, acceleration sensors and brightness sensors.

10. The method according to claim 8, wherein during the automatically identifying step, performing at least one of:

identifying a pattern in the recorded sensor data;

identifying objects contained in the sensor data that are present as an image recording and/or video recording;

identifying a number of existent persons in the image recording and/or the video recording;

identifying an orientation and/or position and/or direction of movement and/or movement of at least one person contained in the image recording and/or the video recording in the image recording and/or the video recording;

identifying a change between at least two scenes in the video recording;

identifying a sound that potentially indicates occurrence of the event to be detected in the sensor data that are present as a sound recording;

identifying a change between at least two scenes in the sound recording;

or

identifying at least one number of human voices contained in the sound recording.

11. The method according to claim 8, wherein:

the set of rating criteria has an acceptance value and/or a rejection value and/or an uncertainty value and/or a base value; and

the acceptance value and/or the rejection value and/or the uncertainty value and/or the base value is acquired and/or modeled during acquisition of the ratings for each of the features.

12. The method according to claim 8, wherein for automatic analysis, performing the following steps:

calculating a respective opinion value for at least one excerpt from the recorded sensor data for the event that is to be detected, for the extracted features on a basis of arithmetic rules corresponding to the set of rating criteria; and

calculating a respective opinion value for at least one of the excerpts from the recorded sensor data for the event that is to be detected for the acquired ratings on a basis of the arithmetic rules corresponding to the set of rating criteria.

13. The method according to claim 12, wherein during the automatically detecting step, identifying at least one of the excerpts from the sensor data, on a basis of the arithmetic rules corresponding to the set of rating criteria and opinion values calculated for the extracted features, for which a probability of occurrence of the event that is to be detected exceeds the probability value calculated from the acquired ratings.