Method and apparatus for providing a scalable multi-camera distributed video processing and visualization surveillance system

• US 7,633,520 B2
• Filed: 06/21/2004
• Issued: 12/15/2009
• Est. Priority Date: 06/19/2003
• Status: Expired due to Fees

• Current Assignee: L-3 Communications Corporation (L3Harris Technologies, Inc.)
• Original Assignee: L-3 Communications Corporation (L3Harris Technologies, Inc.)
• Inventors: Samarasekera, Supun, Arpa, Aydin, Kumar, Rakesh, Paragano, Vincent, Sawhney, Harpreet, Hanna, Keith, Aggarwal, Manoj
• Primary Examiner(s): WONG, ALLEN C
• Application Number:
  US10/872,964
• Publication Number:
  US 20050024206A1
• Time in Patent Office: 2,003 Days
• Field of Search: 348/152, 348/143, 345/427, 345/582, 382/154, 725/131
• US Class Current: 348/152
• CPC Class Codes: G08B 13/19645 Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over
• CPC Class Codes: G08B 13/19689 Remote control of cameras, e.g. remote orientation or image zooming control for a PTZ camera
• CPC Class Codes: G08B 13/19691 Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound

Abstract

A scalable architecture for providing real-time multi-camera distributed video processing and visualization. An exemplary system comprises at least one video capture and storage system for capturing and storing a plurality of input videos, at least one vision based alarm system for detecting and reporting alarm situations or events, and at least one video rendering system (e.g., a video flashlight system) for displaying an alarm situation in a context that speeds up comprehension and response. One advantage of the present architecture is that these systems are all scalable, such that additional sensors (e.g., cameras, motion sensors, infrared sensors, chemical sensors, biological sensors, temperature sensors and like) can be added in large numbers without overwhelming the ability of security forces to comprehend the alarm situation.

Claims

1. A method for monitoring a scene with a computerized surveillance system, said method comprising:

constructing a three dimensional computer model of the scene defining surfaces in the scene being monitored, some of said surfaces corresponding to walls in the scene;

receiving a plurality of input videos each from a respective one of a plurality of cameras monitoring the scene; and

rendering, by a video rendering system, a view of the scene in real time so as to be viewed by a user, said rendering including applying selectively a subset of said plurality of input videos overlaid onto one or more of the surfaces of the three dimensional model of the scene in response to a pose parameter;

detecting whether an alarm situation exists in the scene being monitored and generating an alarm signal when the alarm situation exists; and

selecting, responsive to said alarm signal, said pose parameter so that the rendering is of a view of an area associated with said alarm situation.

2. The method of claim 1, wherein said alarm situation is detected by an alarm detection method.

3. The method of claim 2, wherein said alarm detection method detects motion of objects or new objects within the scene.

4. The method of claim 2, wherein said alarm detection method detects a left behind object within the scene.

5. The method of claim 2, wherein said alarm detection method detects motion of an object in a non-preferred direction within the scene.

6. The method of claim 2, wherein said alarm detection method counts a number of objects within the scene.

7. The method of claim 2, further comprising:

highlighting a portion of the scene to indicate a location associated with said alarm signal.

8. The method of claim 2, wherein said alarm signal is provided by at least one vision based alarm system.

9. The method of claim 1, further comprising:

receiving signals from at least one sensor deployed within the scene.

10. The method of claim 1, wherein said subset of said plurality of input videos is continuously updated to provide a continuous virtual view of the scene.

11. The method of claim 1, wherein said plurality of input videos are provided by a plurality of cameras, wherein at least one of said cameras has pan, tilt and zoom (PTZ) capability.

12. The method of claim 1, wherein said plurality of input videos are provided by a plurality of cameras, wherein at least one of said cameras has pan, tilt and zoom (PTZ) capability, and wherein operation of the PTZ capability of the PTZ camera is controlled by PTZ values generated responsive to the user accessing an interface.

13. The method of claim 1, wherein, responsive to a determination that a viewing angle of one of the input videos from a camera location thereof is sufficiently dissimilar to a viewing angle of the user, said input video is shown in a vicinity of said camera location and not overlaid on said model.

14. The method of claim 1, wherein the pose parameter of the rendering is automatically selected as a virtual viewpoint that best highlights the alarm situation.

15. The method of claim 1, wherein said subset of videos does not include any of the videos that has a view of said surface or surfaces that is occluded by any of the other surfaces of the model.

16. The method of claim 1, further comprising

displaying in the view a status of said alarm situation using a first color before said alarm situation is acknowledged;

and

displaying in the view the status of the alarm situation using a second color different from the first color after said alarm situation is acknowledged.

17. An apparatus for monitoring a scene, said apparatus comprising:

a plurality of cameras providing a plurality of respective input videos;

a vision based alarm system generating an alarm signal when an alarm situation is detected; and

a video rendering system having a pre-existing three-dimensional computer model of the scene having surfaces defined therein, some of said surfaces corresponding to walls of the scene, said video rendering system rendering a view in real time so as to be viewed by a user, the rendering including applying selectively a subset of said plurality of input videos overlaid onto one or more of the surfaces of said three-dimensional computer model of the scene in response to a pose parameter;

said pose parameter being selected based on said alarm signal, so that the rendering is of a view of an area of the model associated with said alarm situation.

18. The apparatus of claim 17, wherein said alarm signal is generated by an alarm detection method.

19. The apparatus of claim 18, wherein said alarm detection method detects motion of objects or new objects within the scene.

20. The apparatus of claim 18, wherein said alarm detection method detects a left behind object within the scene.

21. The apparatus of claim 18, wherein said alarm detection method detects motion of an object in a non-preferred direction within the scene.

22. The apparatus of claim 18, wherein said alarm detection method counts a number of objects within the scene.

23. The apparatus of claim 17, further comprising:

at least one sensor deployed within the scene, said sensor providing a sensor signal to said video rendering system.

24. The apparatus of claim 17, wherein said video rendering system highlights a portion of the scene to indicate a location associated with said alarm signal.

25. The apparatus of claim 17, wherein said subset of said plurality of input videos is continuously updated to provide a continuous bird'"'"'s eye view of the scene.

26. The apparatus of claim 17, wherein at least one of said cameras has pan, tilt and zoom (PTZ) capability.

27. The apparatus of claim 26, wherein, when said pose parameter is selected, a corresponding PTZ value is forwarded to said at least one of said cameras has pan, tilt and zoom (PTZ) capability, and wherein operation of the PTZ capability of the PTZ camera is controlled by PTZ values generated responsive to the user accessing an interface.

28. The apparatus of claim 17, wherein, responsive to a determination that a viewing angle of one of the input videos from a camera location thereof is sufficiently dissimilar to a viewing angle of the user, said input video is shown in a vicinity of said camera location and is not overlaid on said model.

29. The apparatus of claim 17, wherein the pose parameter of the rendering is automatically selected as a virtual viewpoint that best highlights the alarm situation.

30. The apparatus of claim 17, wherein said subset of videos does not include any of the videos that has a view of said surface or surfaces that is occluded by any of the other surfaces of the model.

31. A computer-readable medium having stored thereon a plurality of computer executable instructions that, when executed by a processor, cause the processor to perform the steps of a method for monitoring a scene, said method comprising the steps of:

receiving a plurality of input videos each from a respective one of a plurality of cameras monitoring the scene; and

rendering, by a video rendering system, a view of the scene in real time so as to be viewed by a user, said rendering including accessing a pre-existing three dimensional computer model of the scene, said three dimensional model defining surfaces, some of said surfaces being walls in the scene, and applying selectively a subset of said plurality of input videos overlaid onto one or more of the surfaces of said three dimensional model of the scene in response to a pose parameter;

detecting whether an alarm situation exists in the scene being monitored and generating an alarm signal when the alarm situation exists; and

selecting, responsive to said alarm signal, said pose parameter so that the rendering is of an area associated with said alarm situation.

32. The computer-readable medium of claim 31, wherein the method further comprises:

automatically selecting as the pose parameter a virtual viewpoint that best highlights said alarm situation; and

rendering the view from the pose parameter of said virtual viewpoint.