System and method for user monitoring interface of 3-D video streams from multiple cameras

• US 20070052807A1
• Filed: 09/07/2005
• Published: 03/08/2007
• Est. Priority Date: 09/07/2005
• Status: Active Grant

• Current Assignee: Fujifilm Business Innovation Corp. (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation)
• Inventors: Liu, Qiong, Wilcox, Lynn, Zhou, Hanning, Kimber, Donald, Cooper, Matthew
• Granted Patent: US 7,944,454 B2
• Time in Patent Office: Days
• US Class Current: 348/159
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Embodiments of the present invention introduce a user navigation interface that allows a user to monitor/navigate video streams captured from multiple cameras. It integrates video streams from multiple cameras with the semantic layout into a 3-D immersive environment and renders the video streams in multiple displays on a user navigation interface. It conveys the spatial distribution of the cameras as well as their fields of view and allows a user to navigate freely or switch among preset views. This description is not intended to be a complete description of, or limit the scope of, the invention. Other features, aspects, and objects of the invention can be obtained from a review of the specification, the figures, and the claims.

Claims

1. A system to support 3-D multi-camera video navigation, comprising:

a content analysis module operable to generate a 3-D semantic layout of a site under surveillance;

a rendering module operable to;

integrate video streams from a plurality of cameras with the semantic layout into a 3-D immersive environment; and

render the video streams in a plurality of displays on a user navigation interface; and

the user navigation interface operable to allow a user to perform at least one of;

selecting from a plurality views for each of the plurality of displays; and

tracking a moving object in the video streams with the largest screen resolution of the display in the plurality of displays having the best view of the object.

2. The system according to claim 1, further comprising:

a 3-D layout generation module operable to combine sitemap of the plurality of cameras with the semantic layout.

3. The system according to claim 2, wherein:

the user navigation interface is further operable to allow the user to seamlessly move between content-based and geometry-based mode.

4. The system according to claim 1, further comprising at least one of:

a background subtraction module operable to detect and segment the moving object in the video streams;

a morphological operator operable to filter noise in the video streams from the background subtraction module;

a motion tracking module operable to track the moving object in the video streams; and

a blending module operable to highlight the foreground region of the video streams by alpha blending.

5. The system according to claim 4, wherein:

the blending module is operable to derive alpha values directly from at least one of;

statistics from a foreground/background segmentation method; and

confidence scores from an object detection method.

6. The system according to claim 1, wherein:

the moving object can be a human and/or its face.

7. The system according to claim 1, wherein:

the content analysis module is operable to generate the semantic layout via multidimensional scaling.

8. The system according to claim 1, wherein:

the rendering module is operable to integrate the video streams into the 3-D immersive environment by texture mapping.

9. The system according to claim 1, wherein:

the rendering module is further operable to render orientations of the plurality of displays parallel to the screen at all times on the user navigation interface.

10. The system according to claim 1, wherein:

the user navigation interface is further operable to allow the user to track the location of the moving object.

11. A method to support 3-D multi-camera video navigation, comprising:

.

generating a 3-D semantic layout of a site under surveillance;

integrating video streams from a plurality of cameras with the semantic layout into a 3-D immersive environment;

rendering the video streams in a plurality of displays on a user navigation interface; and

performing at least one of the following operations via the user navigation interface;

selecting from a plurality views for each of the plurality of displays; and

tracking a moving object in the video streams with the largest screen resolution of the display in the plurality of displays having the best view of the object.

12. The method according to claim 11, further comprising:

combining sitemap of the plurality of cameras with the semantic layout.

13. The method according to claim 12, further comprising:

moving seamlessly between content-based and geometry-based mode via the user navigation interface.

14. The method according to claim 11, further comprising at least one of:

detecting and segmenting the moving object in the video streams via background subtraction;

filtering noise in the video streams via morphological operation;

tracking the moving object in the video streams; and

highlighting the foreground region of the video streams by alpha blending.

15. The method according to claim 14, further comprising:

deriving alpha values directly from at least one of;

statistics from a foreground/background segmentation method; and

confidence scores from an object detection method.

16. The method according to claim 11, further comprising at least one of:

generating the semantic layout via multidimensional scaling; and

integrating the video streams into the 3-D immersive environment by texture mapping.

17. The method according to claim 11, further comprising:

rendering orientations of the plurality of displays parallel to the screen at all times on the user navigation interface.

18. The method according to claim 11, further comprising:

tracking the location of the moving object via the user navigation interface.

19. A machine readable medium having instructions stored thereon that when executed cause a system to:

generate a 3-D semantic layout of a site under surveillance;

integrate video streams from a plurality of cameras with the semantic layout into a 3-D immersive environment;

render the video streams in a plurality of displays on a user navigation interface; and

perform at least one of the following operations via the user navigation interface;

selecting from a plurality views for each of the plurality of displays; and

tracking a moving object in the video streams with the largest screen resolution of the display in the plurality of displays having the best view of the object.

20. A system to support 3-D multi-camera video navigation, comprising:

means for generating a 3-D semantic layout of a site under surveillance;

means for integrating video streams from a plurality of cameras with the semantic layout into a 3-D immersive environment;

means for rendering the video streams in a plurality of displays on a user navigation interface; and

means for performing at least one of the following operations via the user navigation interface;

selecting from a plurality views for each of the plurality of displays; and

tracking a moving object in the video streams with the largest screen resolution of the display in the plurality of displays having the best view of the object.