Method and apparatus for providing immersive surveillance
First Claim
Patent Images
1. A surveillance system for monitoring a scene comprising:
- a three-dimensional model of the scene defining surfaces corresponding to surfaces of real objects in the scene;
a plurality of cameras each producing a respective video capturing at least a respective portion of the scene;
an image processor, coupled to said plurality of cameras, constructing a texture map from image data of one or more of the videos and projecting said texture map onto one or more of the surfaces of the model of the scene and rendering a sequence of images representing a view of the scene with the texture map of the image data on said one or more surfaces therein;
a display, coupled to the image processor, displaying the image to a user; and
a view controller, coupled to the image processor, for selecting views of the scene to be displayed; and
wherein the constructing of the texture map includes identifying, for each of said one or more surfaces or portions thereof, a subset of the videos that contain said surface or said portion of said surface; and
wherein said subset of videos does not include any of the videos that has a view of said surface or portion thereof that is occluded by any of the other surfaces of the model.
6 Assignments
0 Petitions
Accused Products
Abstract
A method and apparatus for providing immersive surveillance wherein a remote security guard may monitor a scene using a variety of imagery sources that are rendered upon a model to provide a three-dimensional conceptual view of the scene. Using a view selector, the security guard may dynamically select a camera view to be displayed on his conceptual model, perform a walk through of the scene, identify moving objects and select the best view of those moving objects and so on.
-
Citations
40 Claims
-
1. A surveillance system for monitoring a scene comprising:
-
a three-dimensional model of the scene defining surfaces corresponding to surfaces of real objects in the scene; a plurality of cameras each producing a respective video capturing at least a respective portion of the scene; an image processor, coupled to said plurality of cameras, constructing a texture map from image data of one or more of the videos and projecting said texture map onto one or more of the surfaces of the model of the scene and rendering a sequence of images representing a view of the scene with the texture map of the image data on said one or more surfaces therein; a display, coupled to the image processor, displaying the image to a user; and a view controller, coupled to the image processor, for selecting views of the scene to be displayed; and wherein the constructing of the texture map includes identifying, for each of said one or more surfaces or portions thereof, a subset of the videos that contain said surface or said portion of said surface; and wherein said subset of videos does not include any of the videos that has a view of said surface or portion thereof that is occluded by any of the other surfaces of the model. - View Dependent Claims (2, 3, 4)
-
-
5. A method of providing surveillance of a scene comprising:
-
producing a plurality of videos of the scene; producing a three dimensional model of the scene, said model defining surfaces corresponding to surfaces of real objects in the scene; selecting a viewpoint for the three dimensional model; aligning the plurality of videos with the three dimensional model; rendering said plurality of videos onto the three dimensional model to form a combination view comprising a sequence of output images in which the videos are draped onto the three dimensional model, wherein texture maps on at least some of the surfaces of the three dimensional model are generated from the videos; each of said videos having a resolution relative to each of the surfaces that said video contains, and each of the texture maps for the surfaces being generated by selecting from a subset of the videos each containing the surface the video having the resolution for said surface that is the highest of said resolutions of the videos of the subset; and displaying the combination view. - View Dependent Claims (6, 7, 8, 9, 10, 11)
-
-
12. A system providing surveillance for a site, said system comprising:
-
a computer accessible memory storing data defining a computer-supported three-dimensional model of the site defining one or more surfaces that can be used to produce at least one computer-generated view of the site; a plurality of cameras each having a respective actual view of a respective portion of the site under surveillance, each camera forming a respective video of said actual view, said video comprising electronic signals corresponding to a series of frame images captured by the camera; a computerized image processing system forming a synthetic image of the site as seen from a viewpoint; and a display device providing a display of said synthetic image so as to be viewed by a user; said display of the synthetic image comprising a contextual view of the site wherein at least a first portion of the contextual view is derived from the video from one of the cameras by aligning the video with the model of the site, and wherein texture maps on one or more surfaces of the model of the site are generated from said video, the texture maps on each surface that is contained in a subset of more than one of the videos of said subset of videos being derived from the video containing said surface that is selected as having the highest resolution of said surface, said surfaces corresponding to surfaces of real objects in the site. - View Dependent Claims (13, 14, 15, 16, 17, 18)
-
-
19. A method of providing surveillance of a real-world site, said method comprising:
-
providing a plurality of cameras each transmitting a respective video signal corresponding to a respective real-time view of a respective portion of the site, each video signal comprising signals defining a series of frames of said real-time view, each frame corresponding to said view during a respective time period; receiving said video signals at a computerized image processing system having access to a computer accessible model of the real-world site under surveillance; said computer accessible model being a three-dimensional model of the site and including data defining a plurality of surfaces of the model corresponding to surfaces of real objects of the real-world site; rendering with the image processing system a real time surveillance display comprising a series of images each comprising a synthetic view of the model of the site from a viewpoint; said synthetic view being derived from the model and one or more frames of the video signals selected such that the surveillance display output is a real-time display of the video signals used in the synthetic views; said frames of the video signals being used to generate texture maps of one or more of the surfaces of the three-dimensional model in said rendering, the texture map for each of said surfaces in the model being derived by selecting the video image that, of a subset of the video images each containing the surface, has the highest resolution of the surface; and displaying the surveillance display on a display device so as to be viewable by a user. - View Dependent Claims (20, 21)
-
-
22. A method of providing surveillance of a scene, said method comprising:
-
creating a three-dimensional model of static parts of the scene, said model including a plurality of surfaces corresponding to surfaces of said static parts of the scene; providing video cameras each transmitting respective video data comprising a series of video images each capturing appearance of a portion of the scene at respective time instants; constructing one or more texture maps for the surfaces of the model so as to create a textured model, said texture maps being created by identifying, for one or more portions of the surfaces, a subset of the video images that each contain said portion and that are not occluded by another surface of the model, and projecting one or more color values for the portion of the surface from a pixel or pixels of one of the video images of said subset of video images, wherein said one of said video images has the highest resolution of the surface of said subset of videos; and creating an output image sequence based on the textured model by rendering the textured model so as to form an image thereof as seen from a user-selected virtual viewpoint; wherein the texture maps of the textured model are updated continuously using the incoming video data from the cameras so that the model contains the most currently available video data. - View Dependent Claims (23, 24, 25, 26, 27, 28, 29, 30, 31)
-
-
32. A surveillance system for monitoring a scene comprising:
-
a plurality of cameras producing respective videos each capturing a respective portion of the scene being monitored; a computer system supporting a polyhedral three-dimensional model of the scene having polygonal surfaces defining static objects of the scene corresponding to surfaces of real objects in the scene being monitored; said computer system creating texture maps from incoming video data of the videos and projecting the texture maps onto some of the polygonal surfaces so that the model is a textured model, said texture maps being created by identifying, for one or more portions of a predetermined size of the polygonal surfaces, a subset of the video images that contain said portion and that are not occluded by any other surface of the polyhedral model, and determining one or more color values for the texture map on said portion from a pixel or pixels of one of the video images of said subset, wherein said one of the video images has the highest resolution or most frontal view of said portion of said subset of video images; said computer system rendering the textured model so as to create an output image sequence of output images of the textured model of the scene as seen from a user-selected virtual viewpoint; and an image display displaying the output images to a user; the texture maps of the textured model being updated continuously using the incoming video data from the cameras so that the model contains the most currently available video data. - View Dependent Claims (33, 34, 35, 36, 37, 38, 39)
-
-
40. A surveillance system for monitoring a scene comprising:
-
a plurality of cameras producing respective videos each capturing a respective portion the scene; a computer system supporting a polyhedral three-dimensional model of the scene having polygonal surfaces defining static objects of the scene; said computer system creating texture maps from incoming video data of the videos and projecting the texture maps onto some of the polygonal surfaces so that the model is a textured model, said texture maps being created by identifying, for one or more portions of a predetermined size of the polygonal surfaces, a subset of the video images from one of the time instants that contain said portion and that are not occluded by any other surface of the polyhedral model and determining one or more color values for the texture map on said portion from a pixel or pixels of one of the video images of said subset, wherein said one of the video images has the highest resolution view of said portion of said subset of video images; said computer system having graphics hardware and rendering the textured model so as to create an output images sequence of output images of the textured model of the scene as seen from a user-selected virtual viewpoint; and a viewing screen displaying the output images to a user; the texture maps of the textured model being updated continuously using the incoming video data from the cameras so that the model contains the most currently available video data; and wherein the computer system derives the texture map by selecting a polygonal surface of the model, dividing said selected surface into portions of said predetermined size, selecting one of the portions, detecting by Z-buffering all incoming video images containing said portion and discarding any occluded images so as to obtain said subset of the video images, selecting the video image of said subset that has the highest resolution view of the portion of the surface, and projecting a color value of the portion from the pixels in said highest resolution video image that correspond to the portion.
-
Specification