Method for detecting short term unusual events in videos

• US 7,327,885 B2
• Filed: 06/30/2003
• Issued: 02/05/2008
• Est. Priority Date: 06/30/2003
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Electric Information Technology Center America, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Radhakrishnan, Regunathan, Xiong, Ziyou, Peker, Kadir A., Miyahara, Koji, Divakaran, Ajay
• Primary Examiner(s): Carter; Aaron W
• Application Number:
  US10/610,467
• Publication Number:
  US 20040268380A1
• Time in Patent Office: 1,681 Days
• Field of Search: 382/190, 348/143, 725 37- 60, 725/19
• US Class Current: 382/190
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/58 of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00
• CPC Class Codes: H04H 60/59 of video recognising characters or patterns in general G06F18/00, G06V20/00
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59

Abstract

A method detects short term, unusual events in a video. First, features are extracted features from the audio and the video portions of the video. Segments of the video are labeled according to the features. A global sliding window is applied to the labeled segments to determine global characteristics over time, while a local sliding window is applied only to the labeled segments of the global sliding window to determine local characteristic over time. The local window is substantially shorter in time than the global window. A distance between the global and local characteristic is measured to determine occurrences of the unusual short time events.

Claims

1. A method for detecting short term, unusual events in a video, comprising the steps of:

extracting features from the video;

labeling segments of the video according to the features;

applying a global sliding window to the labeled segments to determine global characteristics over time, while applying a local sliding window to only the labeled segments of the global sliding window to determine local characteristic over time, wherein the local window is substantially shorter in time than the global window and the local sliding window is within the global sliding window; and

measuring a distance between the global and local characteristic to determine occurrences of the unusual short time events.

2. The method of claim 1, further comprising:

performing all steps on a compressed video.

3. The method of claim 1, further comprising:

partitioning the video into an audio portion and a visual portion; and

performing the extracting and labeling, in parallel, on the audio portion and the visual portion to determine audio features and visual features, and audio and visual labels.

4. The method of claim 3, in which the audio labels and the visual labels are synchronized in time.

5. The method of claim 1, in which the features include low, medium, and high level features.

6. The method of claim 1, further comprising:

converting the features to labels by statistical means.

7. The method of claim 1, in which the global window is at least five times longer in time than the local window.

8. The method of claim 1, in which a size of the global window and a size of the local window is learned adaptively from a content of the video.