Method and system for effectively performing event detection in a large number of concurrent image sequences

• US 20050036659A1
• Filed: 07/03/2003
• Published: 02/17/2005
• Est. Priority Date: 07/05/2002
• Status: Active Grant

• Current Assignee: Aspectus Ltd.
• Original Assignee: Aspectus Ltd.
• Inventors: Talmon, Gad, Ashani, Zvi
• Granted Patent: US 8,004,563 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19652 Systems using zones in a single scene defined for different treatment, e.g. outer zone gives pre-alarm, inner zone gives alarm
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: G08B 13/1968 Interfaces for setting up or customising the system
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Method and system for performing event detection and object tracking in image streams by installing in field, a set of image acquisition devices, where each device includes a local programmable processor for converting the acquired image stream that consist of one or more images, to a digital format, and a local encoder for generating features from the image stream. These features are parameters that are related to attributes of objects in the image stream. The encoder also transmits a feature stream, whenever the motion features exceed a corresponding threshold. Each image acquisition device is connected to a data network through a corresponding data communication channel. An image processing server that determines the threshold and processes the feature stream is also connected to the data network. Whenever the server receives features from a local encoder through its corresponding data communication channel and the data network, the server provides indications regarding events in the image streams by processing the feature stream and transmitting these indications to an operator.

Claims

1. Method for performing event detection and object tracking in image streams, comprising:

a) installing in field, a set of image acquisition devices, each of which comprising a local programmable processor for converting the acquired image stream, consisting of one or more images, to a digital format, and a local encoder, for generating, from said image stream, features, being parameters related to attributes of objects in said image stream, and for transmitting a feature stream, whenever said motion features exceed a corresponding threshold;

b) connecting each image acquisition device to a data network through a corresponding data communication channel;

c) connecting an image processing server to said data network, said server being capable of determining said threshold, and of processing said feature stream; and

d) whenever said server receives features from a local encoder through its corresponding data communication channel and said data network, obtaining indications regarding events in said image streams by processing, by said server, said feature stream, and transmitting said indications to an operator.

2. Method according to claim 1, wherein the local encoder is a composite encoder, being the local encoder that further comprises circuitry for compressing the image stream, said composite encoder being capable of operating in a first mode, during which it generates and transmits the features to the server, and in a second mode, during which it transmits to said server, in addition to said features, at least a portion of said image stream in a desired compression level, according to commands sent from said server.

3. Method according to claim 2, further comprising,

controlling each composite encoder, by a command sent from said server, to operate in its first mode;

as long as the server receives features from a composite encoder;

a) controlling that composite encoder, by a command sent from said server, to operate in its second mode; and

b) obtaining indications regarding events in said image streams by processing, by said server, said feature stream, and transmitting said indications and/or their corresponding image streams to an operator.

4. Method according to claim 1, further comprising decoding one or more compressed image streams containing events by said server, and transmitting the decoded image streams to the display of an operator, for viewing.

5. Method according to claim 2, further comprising recording one or more compressed image streams obtained while their local encoder operates in its second mode.

6. Method according to claim 2, further comprising dynamically allocating additional image processing resources, in the server, to data communication channels that receive image streams.

7. Method according to claim 1, wherein one or more feature streams obtained while operating in the first mode, comprises only a portion of the image.

8. Method according to claim 6, further comprising generating and displaying a graphical polygon that encompasses an object of interest, being within the frame of an image or an AOI in said image.

9. Method according to claim 8, further comprising generating and displaying a graphical trace indicating the history of movement of an object of interest, being within the frame of an image or an AOI in said image.

10. Method according to claim 1, wherein the image stream is selected from the group of images that comprises video streams, still images, computer generated images, and pre-recorded digital or analog video data.

11. Method according to claim 1, wherein the image streams are video streams, compressed using MPEG format.

12. Method according to claim 2, wherein during each mode, the encoder uses different resolution and frame rate.

13. Method according to claim 1, wherein the features are selected from the following group:

motion features;

color,

portion of the image;

edge data; and

frequency related information.

14. Method according to claim 1, further comprising performing, by the server, one or more of the following operations and/or any combination thereof:

License Plate Recognition (LPR);

Facial Recognition (FR);

detection of traffic rules violations;

behavior recognition;

fire detection;

traffic flow detection;

smoke detection,

using a feature stream, received from the local encoder of at least one image acquisition device, through its data communication channel

15. System for performing event detection and object tracking in image streams, comprising:

a) a set of image acquisition devices, installed in field, each of which includes;

a.

     1) a local programmable processor for converting the acquired image stream, to a digital format

a.2) a local encoder, for generating, from said image stream, features, being parameters related to attributes of objects in said image stream, and for transmitting a feature stream, whenever said motion features exceed a corresponding threshold;

b) a data network, to which each image acquisition device is connected through a corresponding data communication channel;

c); and

d) an image processing server connected to said data network, said server being capable of determining said threshold, of obtaining indications regarding events in said image streams by processing said feature stream, and of transmitting said indications to an operator.

16. System according to claim 15, in which the local encoder is a composite encoder, being the local encoder that further comprises circuitry for compressing the image stream, said composite encoder being capable of operating in a first mode, during which it generates and transmits the features to the server, and in a second mode, during which it transmits to said server, in addition to said features, at least a portion of said image stream in a desired compression level, according to commands sent from said server.

17. System according to claim 15, further comprising an operator display, for receiving one or more image steams that are decoded by the server and contain events.

18. System according to claim 16, fixer comprising a network video recorder for recording one or more image streams, obtained while their local encoder operates in its first mode.

19. System according to claim 15, in which the server is capable of dynamically allocating additional image processing resources to data communication channels that receive image streams.

20. System according to claim 16, in which one or more image streams obtained while operating in the first mode, comprises only a portion of the image that corresponds to a desired AOI

21. System according to claim 15, in which the server further comprises processing means for generating and displaying a graphical polygon that encompasses an object of interest, being within the frame of an image or an AOI in said image.

22. System according to claim 21, in which the server further comprises processing means for generating and displaying a graphical trace indicating the history of movement of an object of interest, being within the frame of an image or an AOI in said image.

23. System according to claim 15, in which the image stream is selected from the group of images that comprises video streams, still images, computer generated images, and pre-recorded digital or analog video data.

24. System according to claim 15, in which the image streams are video streams, compressed using MPEG format.

25. System according to claim 16, in which during each mode, the encoder uses different resolution and frame rate.

26. System according to claim 15, in which the features are selected from the following group:

motion features;

color;

portion of the image;

edge data; and

frequency related information.

27. System according to claim 15, in which the server further comprises processing means for performing one or more of the following operations and/or any combination thereof:

License Plate Recognition (LPR);

Facial Recognition (FR);

detection of traffic rules violations;

behavior recognition;

fire detection;

traffic flow detection;

smoke detection,

using a feature stream, received from the local encoder of at least one image acquisition device, through its data communication channel.

28. Method for performing event detection and object tracking in image streams, substantially as described and illustrated.

29. System for performing event detection and object tracking in image streams, substantially as described and illustrated.