System and Method for Tracking Moving Objects

• US 20080267451A1
• Filed: 06/22/2006
• Published: 10/30/2008
• Est. Priority Date: 06/23/2005
• Status: Active Grant

• Current Assignee: Israel Aerospace Industries Limited (Government of Israel)
• Original Assignee: Israel Aerospace Industries Limited (Government of Israel)
• Inventors: Karazi, Uri
• Granted Patent: US 8,406,464 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 2207/30212 Military
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A method for tracking an object that is embedded within images of a scene, including: in a sensor unit that includes movable sensor, generating, storing and transmitting over a communication link a succession of images of a scene. In a remote control unit, receiving the succession of images. Receiving a user command for selecting an object of interest in a given image of the received succession of images and determining object data associated with the object and transmitting through the link to the sensor unit the object data. In the sensor unit, identifying the given image of the stored succession of images and the object of interest using the object data, and tracking the object in other image of the stored succession of images. The other image being later than the given image. In the case that the object cannot be located in the latest image of the stored succession of images, using information of at images in which the object was located to predict estimated real-time location of the object and generating direction command to the movable sensor for generating realtime image of the scene and locking on the object.

Claims

1. A method for tracking an object that is embedded within images of a scene, comprising:

(a) in a sensor unit that includes movable sensor, generating, storing and transmitting over a communication link a succession of images of a scene;

(b) in a remote control unit, receiving the succession of images;

(c) receiving a user command for selecting an object of interest in a given image of said received succession of images or derivative thereof, determining object data associated with said object and transmitting through said link to said sensor unit at least said object data;

(d) in the sensor unit, identifying the given image of the stored succession of images and the object of interest using said object data, and tracking the object in at least one other image of said stored succession of images, said other image being later than said given image;

(e) in the case that said object cannot be located in the latest image of said stored succession of images, using information of at least two images in which said object was located to predict estimated real-time location of the object and generating direction command to the movable sensor for generating realtime image of the scene and locking on said object.

2. The method according to claim 1, further comprising using real-time location data for controlling payload means.

3. The method according to claim 1, wherein said payload means includes a designator for illuminating said object.

4. The method according to claim 1, wherein said payload means includes kill mechanism for killing the locked object.

5. The method according to claim 1, wherein said payload means including data collection device for collecting data in respect of the object.

6. The method according to claim 5, wherein said data is collected for intelligence applications.

7. The method according to claim 1, wherein said other images include at least two images or derivative thereof starting from an image that follows said given image.

8. The method according to claim 7, wherein said at least two images are consecutive from among said succession of images or derivative thereof.

9. The method according to claim 1, wherein said given image being the first image in said succession of received images.

10. The method according to claim 1, wherein said succession of images being a sequence of video frames.

11. The method according to claim 1, wherein said data associated with said object includes indication on said given object and at least one of the following:

location of the object, speed of the object, acceleration of the object, size of the object, shape of the object, color of the object.

12. The method according to claim 1, wherein said object is in motion and located in different locations in respective images of said succession of images, and wherein said method further comprising:

generating a derivative image including freezing an image or slowing down its rate, thereby facilitating the selection of said object of interest as if said object'"'"'s location in the images was stationary.

13. The method according to claim 12, further comprising:

pointing in the vicinity of said object thereby zooming the image according to image boundaries substantially defined by said pointing, and selecting said object by pointing thereto on the zoomed image.

14. The method according to claim 13, further comprising enhancing the image before zooming the object.

15. The method according to claim 1, further comprising identifying said object.

16. The method according to claim 1, whereby said object is located in real-time image, notwithstanding a time delay constituted by a timing difference between a first timing that an image is generated in said sensor and a second timing in which said object is located in the real-time image.

18. For use with the method of claim 1, a method for selecting a moving object within images of a scene, comprising:

receiving through a communication link a succession of images;

freezing or slowing down a rate of a given image of said succession of images and selecting an object of interest in the frozen given image, as if said object being stationary;

determining object data associated with said object and transmitting through said link at least said object data.

17. A system that includes a sensor unit that includes movable sensor, the sensor unit is configured to generate a succession of images of a scene, a remote control unit configured to communicate with the sensor unit through a communication link and receive through the link at least said succession of images, and transmit through said link object data that pertain to an object that is embedded in images of the scene, in response to selection of the object in an image;

the system is configured to accomplish substantially real-time location of the object notwithstanding a time delay constituted by a timing difference between a first timing that an image including the object is generated in said sensor and a second timing in which said object is located in real-time image, comprising;

in the control unit, determining object data associated with the object in a given image of said received succession of images or derivative thereof, and transmitting through said link to said sensor unit at least said object data;

in the sensor unit, identifying the given image of the stored succession of images and the object of interest using said object data, and in the case that the object is not located in the latest image of said succession of images, estimating current location of the object, and generating direction commands to said sensor for generating real-time image and locking on said object.

19. A method for selecting a moving object within images of a scene, comprising:

a. receiving through a link a succession of images;

b. freezing or slowing down a rate of a given image of said succession of images and selecting an object of interest in the given image, as if said object is stationary;

c. determining object data associated with said object;

20. The method according to claim 19, further comprising:

transmitting through said link at least said object data.

21. The method according to claims 19, further comprising zooming the frozen image.

22. The method according to claim 19, further comprising enhancing the image.

23. The method according claim 19, wherein said receiving is through a communication link.

24. The method according to claim 19, further comprising:

pointing is in the vicinity of said object thereby zooming the image according to image boundaries substantially defined by said pointing, and selecting said object by pointing thereto on the zoomed image.

25. The method according to claim 19, further comprising:

using the object data for tracking the object.

26. In a system that includes a sensor unit that includes movable sensor, the sensor unit is configured to generate a succession of images of a scene, a remote control unit configured to communicate with the sensor unit through a communication link and receive through the link at least said succession of images, and transmit through said link object data that pertain to an object that is embedded in images of the scene, in response to selection of the object in an image;

the system is configured to accomplish substantially real-time location of the object notwithstanding a time delay constituted by a timing difference between a first timing that an image including the object is generated in said sensor and a second timing in which said object is located in real-time image, comprising;

in the remote unit receiving through the link a succession of images;

freezing or slowing down a rate of a given image of said succession of images and selecting an object of interest in the given image, as if said object is stationary;

and determining object data associated with said object.

27. A method for indicating an area in an image as a candidate of a sought object of interest in a scene, comprising:

receiving data indicative of estimated future coarse location of an object in the scene;

generating a succession of images of an area in the scene, the images includes the estimated coarse location,

comparing between at least two images of said succession, and in the case of difference in at least a portion of respective images that meets a given threshold, the difference is identified as the candidate object.

28. The method according to claim 27, wherein said threshold being one member of a group that includes:

(i) changing of values of at least n pixels in the image, (ii) changing of values of at least n pixels in the image by at least m value delta.

29. The method according to claim 27, wherein said threshold being cluster of pixels whose gray level values were changed.

30. The method according to claim 27, further comprising generating additional at least one image and locating the candidate object in said additional at last one image.

31. The method according to claim 27, further comprising identifying said candidate object as sought object according to at least one following criterion:

location of the object, speed of the object, acceleration of the object, size of the object, shape of the object, color of the object being a cluster of pixels whose gray level value was changed.

32. The method according to claim 27, wherein said coarse location being estimated coordinates of future location of the object.

33. The method according to claim 27, wherein said comparison includes subtraction of images.

34. The method according to claim 27, wherein said comparison includes correlation of images.

35. The method according to claim 27, further comprising applying stabilization techniques to selected images.

36. The method according to claim 27, further comprising, using realtime location data for controlling payload means.

37. The method according to claim 36, wherein said payload means including a designator for illuminating said object.

38. The method according to claim 36, wherein said payload means including kill mechanism for killing the locked object.

39. The method according to claim 36, wherein said payload means including data collection device for collecting data in respect of the object.

40. The method according to claim 39, wherein said data is collected for intelligence applications.

41. A system that includes a sensor unit that includes movable sensor, the sensor unit is configured to generate a succession of images of a scene, a remote control unit configured to communicate with the sensor unit through a communication link and receive through the link at least said succession of images, and transmit through said link object data that pertain to an object that is embedded in images of the scene, in response to selection of the object in an image;

the system is configured to accomplish substantially realtime location of the object notwithstanding a time delay constituted by a timing difference between a first timing that an image including the object is generated in said sensor and a second timing in which said object is located in realtime image, comprising;

in the sensor unit, receiving data indicative of estimated future coarse location of an object in the scene;

generating a succession of images of an area in the scene, the images include the estimated coarse location, and comparing between at least two images of said succession, and in the case of difference in at least a portion of respective images that meets a given threshold, the difference is identified as the candidate object.