Video Surveillance Correlating Detected Moving Objects and RF Signals

• US 20070257985A1
• Filed: 02/27/2007
• Published: 11/08/2007
• Est. Priority Date: 02/27/2006
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Estevez, Leonardo, Gutierrez, Keith
• Granted Patent: US 8,184,154 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19663 Surveillance related processing done local to the camera
• CPC Class Codes: G08B 13/1968 Interfaces for setting up or customising the system
• CPC Class Codes: G08B 13/19695 Arrangements wherein non-video detectors start video recording or forwarding but do not generate an alarm themselves

Abstract

A surveillance method periodically detects an image of the area, identifies and tracks each moving object in a succession of the detected images, detects radio frequency emissions from the area and correlates an identified object with a detected radio frequency emission. The method detects events in the tracking of the moving object. The method stores corresponding data optionally including image data in non-volatile memory upon detection of a combination of an event and a corresponding radio frequency emission. The method triggers an alarm such as an audible alarm, a visual alarm, an email, a short text message or a telephone call detection of a combination of an event and a corresponding radio frequency emission.

Claims

1. A method of monitoring an area, comprising the steps of:

periodically detecting an image of the area;

identifying and tracking a moving object in a succession of the detected images;

detecting radio frequency emissions from objects within the area; and

correlating an identified moving object with a detected radio frequency emission.

2. The method of claim 1, further comprising:

saving one of the detected images as a reference image;

said step of identifying a moving object includes comparing a subsequently detected image to the reference image to identify therein each change region where the subsequently detected image differs from the reference image; and

said step of tracking a moving object includes tracing movement of each change region in successive images.

3. The method of claim 1, further comprising:

detecting an event in the tracking of a moving object.

4. The method of claim 3, wherein:

said step of detecting an event in the tracking of a moving object includes detecting an ENTER event when an object not previously present first appears.

5. The method of claim 3, wherein:

said step of detecting an EXIT event when an object which was previously present is no longer present.

6. The method of claim 3, wherein:

said step of detecting an event in the tracking of a moving object includes detecting a DEPOSIT event when an object splits into two objects, one of which is moving and the other of which is stationary.

7. The method of claim 3, wherein:

said step of detecting an event in the tracking of a moving object includes detecting a REMOVE event when a moving object merges with a stationary object and continues to move while the stationary object disappears.

8. The method of claim 3, wherein:

said step of detecting an event in the tracking of a moving object includes detecting a REST event when a moving object comes to a stop and continues to be present without moving.

9. The method of claim 3, wherein:

said step of detecting an event in the tracking of a moving object includes detecting a MOVE event when a detected object which has been stationary begins moving again.

10. The method of claim 3, further comprising:

storing data in non-volatile memory upon detection of a predetermined combination of an event and a predetermined correlation to a detected radio frequency emission.

11. The method of claim 10, further comprising:

storing image data in non-volatile memory data corresponding to said data stored upon detection of the predetermined event.

12. The method of claim 11, wherein:

said step of tracking the moving object includes

forming a bounding box just large enough to include each detected object in each image;

comparing a temporary bounding box of each detected object with a corresponding bounding box of each image,

temporarily storing a larger of the temporary bounding box or the corresponding bounding box of a current image; and

said step of storing image data in non-volatile memory includes storing the temporary bounding box in non-volatile memory.

13. The method of claim 10, wherein:

said step of tracking the moving object includes temporarily storing track data indicative of the track of the moving object; and

said step of storing data in non-volatile memory includes storing said track data in non-volatile memory.

14. The method of claim 10, further comprising:

recalling stored data corresponding to user specified parameters.

15. The method of claim 14, further comprising:

storing image data in non-volatile memory corresponding to said data stored upon detection of the predetermined event; and

wherein said step of recalling stored data includes recalling and viewing stored image data.

16. The method of claim 14, further comprising:

said step of tracking the moving object includes

forming a bounding box just large enough to include each detected object in each image;

comparing a temporary bounding box of each detected object with a corresponding bounding box of each image,

temporarily storing a larger of the temporary bounding box or the corresponding bounding box of a current image;

said step of storing image data in non-volatile memory includes storing the temporary bounding box in non-volatile memory; and

wherein said step of recalling stored data includes recalling and viewing said stored bounding box.

17. The method of claim 14, further comprising:

said step of tracking the moving object includes temporarily storing track data indicative of the track of the moving object;

said step of storing data in non-volatile memory includes storing said track data in non-volatile memory; and

wherein said step of recalling stored data includes recalling and viewing said stored track data.

18. The method of claim 3, further comprising:

storing data in non-volatile memory upon detection of a predetermined combination of an event and no correlation to any detected radio frequency emission.

19. The method of claim 18, further comprising:

storing image data in non-volatile memory data corresponding to said data stored upon detection of the predetermined event.

20. The method of claim 19, wherein:

said step of tracking the moving object includes

forming a bounding box just large enough to include each detected object in each image;

comparing a temporary bounding box of each detected object with a corresponding bounding box of each image,

temporarily storing a larger of the temporary bounding box or the corresponding bounding box of a current image; and

said step of storing image data in non-volatile memory includes storing the temporary bounding box in non-volatile memory.

21. The method of claim 19, wherein:

said step of tracking the moving object includes temporarily storing track data indicative of the track of the moving object; and

said step of storing data in non-volatile memory includes storing said track data in non-volatile memory.

22. The method of claim 19, further comprising:

recalling stored data corresponding to user specified parameters.

23. The method of claim 22, further comprising:

storing image data in non-volatile memory corresponding to said data stored upon detection of the predetermined event; and

wherein said step of recalling stored data includes recalling and viewing stored image data.

24. The method of claim 22, further comprising:

said step of tracking the moving object includes

forming a bounding box just large enough to include each detected object in each image;

comparing a temporary bounding box of each detected object with a corresponding bounding box of each image,

temporarily storing a larger of the temporary bounding box or the corresponding bounding box of a current image;

said step of storing image data in non-volatile memory includes storing the temporary bounding box in non-volatile memory; and

wherein said step of recalling stored data includes recalling and viewing said stored bounding box.

25. The method of claim 22, further comprising:

said step of tracking the moving object includes temporarily storing track data indicative of the track of the moving object;

said step of storing data in non-volatile memory includes storing said track data in non-volatile memory; and

wherein said step of recalling stored data includes recalling and viewing said stored track data.

26. The method of claim 3, further comprising:

triggering an alarm upon detection of a predetermined combination of an event and a predetermined correlation to a detected radio frequency emission.

27. The method of claim 26, wherein:

said step of triggering an alarm includes triggering an audible alarm.

28. The method of claim 26, wherein:

said step of triggering an alarm includes triggering a visual alarm.

29. The method of claim 26, wherein:

said step of triggering an alarm includes transmitting an email to a predetermined remote location.

30. The method of claim 26, wherein:

said step of triggering an alarm includes transmitting a telephone call to a predetermined telephone.

31. The method of claim 26, wherein:

said step of triggering an alarm includes transmitting short text message to a predetermined portable telephone.

32. The method of claim 3, further comprising:

triggering an alarm upon detection of a predetermined combination of an event and no correlation to any detected radio frequency emission.

33. The method of claim 32, wherein:

said step of triggering an alarm includes triggering an audible alarm.

34. The method of claim 32, wherein:

said step of triggering an alarm includes triggering a visual alarm.

35. The method of claim 32, wherein:

said step of triggering an alarm includes transmitting an email to a predetermined remote location.

36. The method of claim 32, wherein:

said step of triggering an alarm includes transmitting a telephone call to a predetermined telephone.

37. The method of claim 32, wherein:

said step of triggering an alarm includes transmitting short text message to a predetermined portable telephone.

38. The method of claim 1, wherein:

said step of detecting radio frequency emissions includes detecting WIFI radio frequency emissions.

39. The method of claim 1, wherein:

said step of detecting radio frequency emissions includes detecting Bluetooth radio frequency emissions.

40. The method of claim 1, wherein:

said step of detecting radio frequency emissions includes

generating a radio frequency identification interrogation signal, and

detecting a radio frequency identification return signal.

41. The method of claim 1, wherein:

said of correlating the identified moving object with the detected radio frequency emission includes correlating an identified moving object growing larger with time with a detected radio frequency emission growing larger with time.

42. The method of claim 1, wherein:

said of correlating the identified moving object with the detected radio frequency emission includes correlating an identified moving object growing smaller with time with a detected radio frequency emission growing smaller with time.

43. The method of claim 1, wherein:

said of correlating the identified moving object with the detected radio frequency emission includes correlating an identified moving object upon initial identification with a detected radio frequency emission upon initial detection.

44. The method of claim 1, wherein:

said step of detecting radio frequency emissions from objects within the area includes detecting a RF emission with a directionality encompassing a sub-area less than all of the area; and

said step of correlating the identified moving object with the detected radio frequency emission includes correlating an identified moving object within said sub-area with said detected radio frequency emission.

45. The method of claim 1, wherein:

said step of periodically detecting an image of the area includes controlling pan, tilt and zoom of a camera; and

said step of tracking the moving object includes detecting the moving object within a frame of the image of the camera and correcting a track of the moving object dependent upon a pan, tilt and zoom state of the camera.

46. The method of claim 1, further comprising:

training said correlation of the identified moving object with the detected radio frequency emission by introducing an object with a known RF emission into the area.