System and method for user monitoring interface of 3-D video streams from multiple cameras

• US 7,944,454 B2
• Filed: 09/07/2005
• Issued: 05/17/2011
• Est. Priority Date: 09/07/2005
• Status: Active Grant

• Current Assignee: Fujifilm Business Innovation Corp. (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Inventors: Cooper, Matthew L., Zhou, Hanning, Kimber, Donald, Liu, Qiong, Wilcox, Lynn
• Primary Examiner(s): SAMS, MICHELLE L
• Application Number:
  US11/220,730
• Publication Number:
  US 20070052807A1
• Time in Patent Office: 2,078 Days
• Field of Search: 348/170, 348/169, 382/173, 345/629, 345/638, 345/170, 715/243, 715/782
• US Class Current: 345/629
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Embodiments of the present invention introduce a user navigation interface that allows a user to monitor/navigate video streams captured from multiple cameras. It integrates video streams from multiple cameras with the semantic layout into a 3-D immersive environment and renders the video streams in multiple displays on a user navigation interface. It conveys the spatial distribution of the cameras as well as their fields of view and allows a user to navigate freely or switch among preset views. This description is not intended to be a complete description of, or limit the scope of, the invention. Other features, aspects, and objects of the invention can be obtained from a review of the specification, the figures, and the claims.

Claims

1. A video surveillance system coupled with a plurality of cameras, comprising:

one or more microprocessors;

memory storing instructions for execution by the one or more microprocessors, the instructions including;

a content analysis module operable to generate a 3-D layout of a site under surveillance, the 3-D layout representing respective locations of a plurality of polygons for mapping video streams from the plurality of cameras;

a rendering module operable to render one or more video streams by mapping respective video streams from respective cameras onto respective polygons represented by the 3-D layout; and

a user navigation interface operable to display the one or more video streams.

2. The system according to claim 1, further comprising:

a 3-D layout generation module operable to update the 3-D layout based at least on sitemap of the plurality of cameras and the layout.

3. The system according to claim 2, wherein:

the user navigation interface is further operable to allow the user to select between a content-based mode and a geometry-based mode.

4. The system according to claim 1, further comprising at least one of:

a background subtraction module operable to detect and segment the moving object in the video streams;

a morphological operator operable to filter noise in the video streams from the background subtraction module;

a motion tracking module operable to track the moving object in the video streams; and

a blending module operable to highlight the foreground region of the video streams by alpha blending.

5. The system according to claim 4, wherein:

the blending module is operable to derive alpha values directly from at least one of;

statistics from a foreground/background segmentation method; and

confidence scores from an object detection method.

6. The system according to claim 1, wherein:

the moving object includes a human and/or its face.

7. The system according to claim 1, wherein:

the content analysis module is operable to generate the 3-D layout in accordance with a multidimensional scaling method based on information from the plurality of cameras.

8. The system according to claim 1, wherein:

the rendering module is operable to render the one or more video streams by texture mapping the respective video streams from the respective cameras onto the respective polygons.

9. The system according to claim 1, wherein:

the rendering module is further operable to render orientations of the plurality of polygons parallel to a display screen at all times on the user navigation interface.

10. The system according to claim 1, wherein:

the user navigation interface is further operable to allow a user to track a respective location of a respective moving object.

21. The system of claim 1, wherein the user navigation interface is operable to:

receive an input from a user to track a moving object in the video streams; and

display a video stream having a best view of the object or a video stream with a largest screen resolution having a view of the object.

11. A method to support 3-D multi-camera video navigation, at a video surveillance system coupled with a plurality of cameras, the video surveillance system including one or more microprocessors and memory storing instructions for execution by the one or more microprocessors, the method comprising:

generating a 3-D layout of a site under surveillance, the 3-D layout representing respective locations of a plurality of polygons for mapping video streams from the plurality of cameras;

rendering one or more video streams by mapping respective video streams from respective cameras onto respective polygons represented by the 3-D layout; and

displaying the one or more video streams.

12. The method according to claim 11, further comprising:

updating the 3-D layout based at least on sitemap of the plurality of cameras and the layout.

13. The method according to claim 12, further comprising:

receiving a user selection between a content-based mode and a geometry-based mode via the user navigation interface.

14. The method according to claim 11, further comprising at least one of:

detecting and segmenting the moving object in the video streams via background subtraction;

filtering noise in the video streams via morphological operation;

tracking the moving object in the video streams; and

highlighting the foreground region of the video streams by alpha blending.

15. The method according to claim 14, further comprising:

deriving alpha values directly from at least one of;

statistics from a foreground/background segmentation method; and

confidence scores from an object detection method.

16. The method according to claim 11, further comprising at least one of:

generating the layout in accordance with a multidimensional scaling method based on information from the plurality of cameras; and

rendering the one or more video streams by texture mapping the respective video streams from the respective cameras onto the respective polygons.

17. The method according to claim 11, further comprising:

rendering orientations of the plurality of polygons parallel to a display screen at all times on the user navigation interface.

18. The method according to claim 11, further comprising:

tracking a respective location of a respective moving object via the user navigation interface.

19. A non-transitory machine readable medium having instructions stored thereon that when executed by a video surveillance system cause the video surveillance system to:

generate a 3-D layout of a site under surveillance, the 3-D layout representing respective locations of a plurality of polygons for mapping video streams from the plurality of cameras;

render one or more video streams by mapping respective video streams from respective cameras onto respective polygons represented by the 3-D layout; and

displaying the one or more video streams.

20. A video surveillance system coupled with a plurality of cameras, comprising:

means for generating a 3-D layout of a site under surveillance, the 3-D layout representing respective locations of a plurality of polygons for mapping video streams from the plurality of cameras;

means for rendering one or more video streams by mapping respective video streams from respective cameras onto respective polygons represented by the 3-D layout; and

means for displaying the one or more video streams.