Motion based event detection system and method

• US 6,424,370 B1
• Filed: 10/08/1999
• Issued: 07/23/2002
• Est. Priority Date: 10/08/1999
• Status: Expired due to Term

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Courtney, Jonathan D.
• Primary Examiner(s): DIEP, NHON THANH
• Application Number:
  US09/414,610
• Time in Patent Office: 1,019 Days
• Field of Search: H04/N.718, 348/152, 348/155, 348/143, 348/135, 348/142, 348/154, 348/171, 348/172, 348/169, 382/103, 382/107, 382/194, 382/236, 375/240.13, 375/240.15, 375/240.17, 707/100, 707/102, 707/103, 707/104
• US Class Current: 348/143
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

A method to provide automatic content-based video indexing from object motion is described. Moving objects in video from a surveillance camera 11 detected in the video sequence using motion segmentation methods by motion segmentor 21. Objects are tracked through segmented data in an object tracker 22. A symbolic representation of the video is generated in the form of an annotated graphics describing the objects and their movement. A motion analyzer 23 analyzes results of object tracking and annotates the graph motion with indices describing several events. The graph is then indexed using a rule based classification scheme to identify events of interest such as appearance/disappearance, deposit/removal, entrance/exit, and motion/rest of objects. Clips of the video identified by spatio-temporal, event, and object-based queries are recalled to view the desired video.

Claims

1. A method of providing video indexing comprising the steps of:

(a) detecting objects in video to provide detected objects comprising the step of performing motion segmentation including image differencing from a reference image and thresholding and storing meta-information in a database where said meta-information is in the form of a directed graph;

(b) analyzing motion of said detected objects including tracking said detected objects using linear prediction of object position;

(c) generating a symbolic motion description of object motion; and

(d) placing index marks in said symbolic motion description to identify occurrence of events in the video.

2. The method of claim 1 wherein the step of said placing index marks identifies occurrences of appearance, disappearance, deposit, removal, entrance, exit, motion or rest of objects.

3. The method of claim 1 including the steps of:

(a) receiving content-based queries;

(b) matching queries with symbolic video information and said index marks; and

(c) providing video sequences corresponding to the query.

4. The method of claim 3 wherein constraints for queries are specified by manipulation of video clips.

5. The method of claim 1 wherein said motion segmentation includes connected component analysis.

6. The method of claim 1 wherein said image differencing includes applying an edge detector operator to the current image and the reference image and checking for coincident boundary pixels in the segmented region.

7. The method of claim 1 wherein said detected objects in video are from an IR sensor and said image differencing compares the variance of pixel intensities within the region of interest of the current image and reference image.

8. The method of claim 1 wherein said motion segmentation includes morphological close operation.