Method and apparatus for automatic visual detection, recording, and retrieval of events

• US 20050276445A1
• Filed: 02/16/2005
• Published: 12/15/2005
• Est. Priority Date: 06/09/2004
• Status: Abandoned Application

• Current Assignee: Cognex Technology & Investment Corporation (Cognex Corporation)
• Original Assignee: Cognex Technology & Investment Corporation (Cognex Corporation)
• Inventors: Phillips, Brian S., Silver, William M.
• Application Number:
  US11/059,512
• Publication Number:
  US 20050276445A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G01N 2021/8887 based on image processing techniques
• CPC Class Codes: G06T 7/0008 checking presence/absence
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 5/907 using static stores, e.g. storage tubes or semiconductor memories H04N5/91 takes precedence
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

Disclosed are methods and apparatus for automatic visual detection of events, for recording images of those events and retrieving them for display and human or automated analysis, and for sending synchronized signals to external equipment when events are detected. An event corresponds to a specific condition, among some time-varying conditions within the field of view of an imaging device, that can be detected by visual means based on capturing and analyzing digital images of a two-dimensional field of view in which the event may occur. Events may correspond to rare, short duration mechanical failures for which obtaining images for analysis is desirable. Events are detected by considering evidence obtained from an analysis of multiple images of the field of view, during which time moving mechanical components can be seen from multiple viewing perspectives.

Claims

1. A method for automatic visual detection, recording, and retrieval of an event, comprising:

capturing a sequence of frames, each frame in the sequence of frames comprising an image of a field of view in which the event occurs, the event corresponding to a specific condition, among time-varying conditions within the field of view, that can be detected by visual means;

analyzing each frame of the sequence of frames to obtain evidence that the event has occurred and to identify at least one event frame, the analysis of the at least one event frame revealing sufficient evidence that the event has occurred;

choosing a plurality of selected frames from the sequence of frames, such that the step of choosing to include a frame from the sequence of frames in the plurality of selected frames is responsive to the position of the frame in the sequence of frames relative to the at least one event frame;

recording the plurality of selected frames in a memory;

receiving commands to retrieve frames from the recorded plurality of selected frames; and

retrieving frames from the recorded plurality of selected frames in response to the commands.

2. The method of claim 1, wherein:

the commands are responsive to an activity of a human user; and

the retrieved frames are displayed for the human user.

3. The method of claim 1, wherein at least one of the frames in the plurality of selected frames occurs prior to the at least one event frame in the sequence of frames.

4. The method of claim 1, wherein at least one of the frames in the plurality of selected frames occurs after the at least one event frame in the sequence of frames.

5. The method of claim 1, wherein:

at least one of the frames in the plurality of selected frames occurs prior to the at least one event frame in the sequence of frames; and

at least one of the frames in the plurality of selected frames occurs after the at least one event frame in the sequence of frames.

6. The method of claim 1, wherein:

the commands are responsive to an automated image analysis system; and

the retrieved frames are analyzed by the automated image analysis system.

7. The method of claim 2, further comprising:

displaying a plurality of retrieved frames as a corresponding plurality of low-resolution “

thumbnail”

images; and

displaying at high-resolution at least one retrieved frame corresponding to at least one of the plurality of low-resolution “

thumbnail”

images.

8. The method of claim 1, wherein the analyzing step further comprises:

analyzing each frame in the sequence of frames to determine a corresponding sequence of event detection weights, each event detection weight in the sequence of event detection weights comprising evidence that the event has occurred in the corresponding frame; and

analyzing the sequence of event detection weights to identify the at least one event frame, such that a combined evidence responsive to the event detection weights corresponding to the at least one event frame reveals sufficient evidence that the event has occurred.

9. The method of claim 8 wherein:

the at least one event frame comprises a plurality of event frames; and

the combined evidence is responsive to all of the event detection weights corresponding to the plurality of event frames.

10. A system for automatic visual detection, recording, and retrieval of events, comprising:

a capture process that captures a sequence of frames, each frame in the sequence of frames comprising an image of a field of view in which the event occurs, the event corresponding to a specific condition, among time-varying conditions within the field of view, that can be detected by visual means;

an analysis process that analyzes each frame of the sequence of frames to obtain evidence that the event has occurred and to identify at least one event frame, the analysis of the at least one event frame revealing sufficient evidence that the event has occurred;

a selection process that chooses a plurality of selected frames from the sequence of frames, such that the selection process chooses to include a frame from the sequence of frames in the plurality of selected frames responsive to the position of the frame in the sequence of frames relative to the at least one event frame;

a memory that records the plurality of selected frames; and

an interface that receives commands to retrieve frames from the recorded plurality of selected frames and that retrieves frames from the recorded plurality of selected frames in response to the commands.

11. The system of claim 10, wherein the interface includes a human-machine interface, and wherein:

the commands are responsive to an activity of a human user with the human-machine interface; and

the interface is adapted to display retrieved frames for the human user.

12. The system of claim 10, wherein at least one of the frames in the plurality of selected frames occurs prior to the active frame in the sequence of frames.

13. The system of claim 10, wherein at least one of the frames in the plurality of selected frames occurs after the active frame in the sequence of frames.

14. The system of claim 10, wherein:

at least one of the frames in the plurality of selected frames occurs prior to the active frame in the sequence of frames; and

at least one of the frames in the plurality of selected frames occurs after the active frame in the sequence of frames.

15. The system of claim 10, wherein the interface includes an automated image analysis system, and wherein:

the commands are responsive to the automated image analysis system; and

the retrieved frames are analyzed by the automated image analysis system.

16. The system of claim 11, wherein the interface includes:

a display adapted to display a plurality of retrieved frames as a corresponding plurality of low-resolution “

thumbnail”

images, the display being adapted to allow selection, responsive to the human user, of at least one “

thumbnail”

image from the plurality of “

thumbnail”

images; and

a high-resolution display adapted to display, at high-resolution, at least one retrieved frame corresponding to the at least one “

thumbnail”

image.