Object detection method and system for scene change analysis in TV and IR data

• US 6,049,363 A
• Filed: 02/05/1997
• Issued: 04/11/2000
• Est. Priority Date: 02/05/1996
• Status: Expired due to Term

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Nair, Dinesh R., Courtney, Jonathan D.
• Primary Examiner(s): Britton, Howard
• Assistant Examiner(s): VO, TUNG T
• Application Number:
  US08/795,434
• Time in Patent Office: 1,161 Days
• Field of Search: 348/135, 348/148, 348/42, 348/700, 348/699, 348/385, 348/118, 348/576-579, 348/162-172, 348/25, 340/435, 340/937, 340/152, 340/155, 358/22, 382/107
• US Class Current: 348/700
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 16/7854 using shape G06F16/7837 takes precedence
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression
• CPC Class Codes: H04N 5/147 Scene change detection
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

Object detection for scene change analysis is performed by a statistical test applied to data extracted from two images taken from the same scene from identical viewpoints. It is assumed that a single change region corresponding to an object that is present in one image but absence in the other is given. In the case of TV data, the test consists of measuring the coincidence of edge pixels in each image with the boundary of the change region. In the case of IR data, the tests consist of measuring the pixel intensity variance within the change region in each image.

Claims

1. A method of performing object detection for scene change analysis in Infrared camera data comprising the steps of:

a) acquiring two Infrared camera images of a scene from identical view points at different times;

b) determining a change region corresponding to an object that is present in one image but absent in the other image;

c) applying a statistical test to the two images within the change region to detect objects;

d) analyzing motion of said objects;

e) generating a symbolic motion description of object motion; and

f) placing index marks in said symbolic motion description to identify occurrence of events in the video.

2. The method of claim 1 wherein the statistical test comprises measuring the pixel intensity variance within the change region in each image.

3. The method of claim 2 wherein said analyzing step includes tracking said detected objects using linear prediction of object position.

4. The method of claim 2 wherein said analyzing step includes associating objects in successive frames of said video using a mutual-nearest-neighbor criterion.

5. The method of claim 2 wherein said analyzing step includes determining paths of said objects and intersection with paths of other objects.

6. The method of claim 2 wherein said generating step includes generating a directed graph reflective of the paths and path intersections of said objects.

7. The method of claim 2 wherein said generating step includes generating primary and secondary graph links reflective of the likelihood of accurate motion analysis.

8. The method of claim 2 wherein said generating step includes generating a record of image statistics for objects in every video frame.

9. The method of claim 8 wherein said generating a record of image statistics step includes generating size, shape, position, and time-stamp of objects in every video frame.

10. The method of claim 2 wherein said generating step includes generating hierarchical graph node groupings reflective of paths and intersections of said objects.

11. The method of claim 10 wherein said hierarchical graph node groupings reflect the likelihood of accurate motion analysis.

12. The method of claim 2 wherein said placing index marks step includes placement of index marks corresponding to motion events in accordance with said hierarchical graph node groupings.

13. The method of claim 12 wherein said placing index marks step includes use of a rule-based classifier.

14. The method of claim 12 wherein said placing index marks step corresponding to motion events includes placement of one or more marks corresponding to appearance, disappearance, deposit, removal, entrance, exit, motion, or rest of objects.

15. A method of performing object detection for scene change in video IR data comprising the steps of:

a) acquiring two IR images of a scene from identical view points at different items;

b) determining a change region corresponding to an object that is present in one image but absent in the other image;

c) applying a statistical test to the two images within the change region to detect objects;

d) analyzing motion of said objects;

e) generating a symbolic motion description of object motion;

f) placing index marks in said symbolic motion description to identify occurrence of events in video;

g) receiving content-base queries;

h) matching queries with symbolic video information and said index marks; and

i) providing video sequences corresponding to the query.

16. The method of claim 15 wherein the statistical test comprises measuring the pixel intensity variance within the change region in each image.

17. The method of claim 15 wherein said step of receiving content-base queries includes receiving queries with constraints involving one or more of a video clip, time interval, object, spatial region, or motion event.

18. The method of claim 17 wherein constraints for said queries can be specified by manipulation of video clips.

19. The method of claim 15 wherein said step of matching queries with symbolic video information includes filtering symbolic information to meet the query constraints of one or more video clip, time interval, object, spatial region, or motion event.

20. The method of claim 15 wherein said step of providing video sequences includes a system clipboard with sets of video clips for progressive refinement of content based queries and query results.

21. A system for providing video indexing of IR images comprising:

a) means for acquiring two IR images of a scene from identical view points at different times;

b) means for determining a change region corresponding to an object that is present in one image but absent in the other image;

c) means for determining pixel intensity variance within the change region to detect objects;

d) means for analyzing motion of said objects;

e) means for generating a symbolic motion description of object motion; and

f) means for placing index marks in said symbolic motion description to identify occurrence of events in the video.