Method and apparatus for providing immersive surveillance

• US 20030085992A1
• Filed: 07/24/2002
• Published: 05/08/2003
• Est. Priority Date: 03/07/2000
• Status: Active Grant

• Current Assignee: L-3 Communications Corporation (L3Harris Technologies, Inc.)
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Hsu, Stephen, Samarasekera, Supun, Arpa, Aydin, Kumar, Rakesh, Hanna, Keith, Aggarwal, Manoj, Sawhney, Harpreet Singh, Nister, David
• Granted Patent: US 7,522,186 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19641 Multiple cameras having overlapping views on a single scene
• CPC Class Codes: G08B 13/19693 using multiple video sources viewed on a single or compound screen
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A method and apparatus for providing immersive surveillance wherein a remote security guard may monitor a scene using a variety of imagery sources that are rendered upon a model to provide a three-dimensional conceptual view of the scene. Using a view selector, the security guard may dynamically select a camera view to be displayed on his conceptual model, perform a walk through of the scene, identify moving objects and select the best view of those moving objects and so on.

Claims

1. A surveillance system for monitoring a scene comprising:

a plurality of cameras producing video capturing the scene;

an image processor, coupled to said plurality of cameras, for applying each of the videos to a model of the scene;

a display, coupled to the image processor, for displaying a combination of the videos and model; and

a view controller, coupled to the image processor, for selecting views of the combination to be displayed.

2. The surveillance system of claim 1 wherein the image processor further comprises:

a three-dimensional model generator; and

an image rendering processor.

3. The surveillance system of claim 2 wherein said image processor further comprises:

means for identifying moving objects in each of the videos;

means for producing an icon representing each of the identified moving objects;

means for inserting the icon into the combination.

4. The surveillance system of claim 3 further comprising:

means for identifying a person as a moving object; and

means for annotating the display with a description of the person.

5. A display of a monitored scene comprising:

a three dimensional model of a scene having a plurality of video images rendered onto the three dimensional model.

6. The display of claim 5 wherein each of the video images in the plurality of video images is produced by a different camera viewing the scene from different locations.

7. The display of claim 5 further comprising icons representing moving objects identified within the plurality of video images.

8. The display of claim 5 further comprising a region displaying each of the video images as produced by a plurality of cameras.

9. The display of claim 5 wherein a viewpoint of the three dimensional model is selectable.

10. The display of claim 9 wherein the viewpoint is selected as a viewpoint of one of a plurality of cameras that produces the video images.

11. The display of claim 9 wherein the viewpoint is any location in three-dimensional space.

12. The display of claim 7 wherein the icon is annotated with text identifying the moving object.

13. A display of a monitored scene comprising:

a selectable view of the scene, wherein said view is one of a three dimensional model of a scene having a plurality of video images rendered onto the three dimensional model;

a bird'"'"'s eye view of the scene;

or a flashlight view of the scene.

14. The display of claim 13 wherein said bird'"'"'s eye view comprises icons representing moving objects, where each object icon is color coded in a color that matches a color of a camera icon representing a camera that views the moving object.

15. The display of claim 14 wherein said object icon comprises a shadow indicating prior positions of the moving object.

16. The display of claim 14 wherein the flashlight view is selected by selecting a particular location in the three dimensional model of bird'"'"'s eye view.

17. A method of providing surveillance of a scene comprising:

producing a plurality of videos of a scene;

producing a three dimensional model of the scene;

selecting a viewpoint for the three dimensional model;

aligning the plurality of videos with the three dimensional;

rendering said plurality of videos onto the three dimensional model to form a combination view; and

displaying the combination view.

18. The method of claim 17 wherein the selected viewpoint is a bird'"'"'s eye view.

19. The method of claim 17 further comprising:

selecting a location within the combination view; and

producing a view of the scene from the selected location.

20. The method of claim 17 further comprising:

identifying moving objects in the videos;

producing an icon representing each moving object; and

inserting the icon into the combination view.

21. The method of claim 17 further comprising:

storing the combination view for replay.

22. The method of claim 20 wherein said identifying step further comprises:

identifying the object as a person; and

annotating the combination view with text representing the identification of the person.

23. A method of providing surveillance of a scene comprising:

producing a plurality of videos of a scene;

producing an implicit 3D model of the scene;

selecting a region of a displayed video and using the 3D model to identify another video to be displayed to best produce an image of the selected region of the first video.