Visualizing camera feeds on a map

• US 8,237,791 B2
• Filed: 03/19/2008
• Issued: 08/07/2012
• Est. Priority Date: 03/19/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Chen, Billy, Ofek, Eyal
• Primary Examiner(s): Winder, Patrice
• Assistant Examiner(s): Golabbakhsh, Ebrahim
• Application Number:
  US12/050,944
• Publication Number:
  US 20090237510A1
• Time in Patent Office: 1,602 Days
• Field of Search: 348/159, 348/149, 348/143, 348/48, 348/51, 348/52, 348/160, 345/419, 701/468
• US Class Current: 348/143
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors

Abstract

Feeds from cameras are better visualized by superimposing images based on the feeds onto map based on a two- or three-dimensional virtual map. For example, a traffic camera feed can be aligned with a roadway included in the map. Multiple videos can be aligned with roadways in the map and can also be aligned in time.

Claims

1. A computer-readable storage medium that does not consist of a signal, said computer-readable storage medium storing computer-executable instructions that, when executed, cause a computing device to perform a method for visualizing camera feeds, said method comprising:

displaying information that identifies a plurality of available camera feeds;

receiving a user selection identifying a first selected camera feed and a second selected camera feed;

accessing said first selected camera feed and said second selected camera feed from said plurality of available camera feeds;

accessing data for rendering a map display of a multi-dimensional virtual map;

associating said first selected camera feed and said second selected camera feed with corresponding regions included in said map display of said multi-dimensional virtual map based on said data;

superimposing one or more frames of said first selected camera feed and one or more frames of said second selected camera feed onto said corresponding regions included in said map display of said multi-dimensional virtual map, wherein said one or more frames of said first selected camera feed and said one or more frames of said second selected camera feed are aligned with respect to time; and

rendering said map display of said multi-dimensional virtual map with said one or more frames of said first selected camera feed and said one or more frames of said second selected camera feed being overlaid on said corresponding regions and with a remainder of said map display of said multi-dimensional virtual map being displayed around said corresponding regions.

2. The computer-readable storage medium of claim 1 wherein:

said first selected camera feed comprises a dynamic video feed of vehicles traveling on a roadway,

said map display of said multi-dimensional virtual map includes an image of said roadway, and

said method further comprises aligning said dynamic video feed with said image of said roadway.

3. The computer-readable storage medium of claim 1 wherein said method further comprises:

tracking location and speed of said vehicles to compute positions of said vehicles with respect to time.

4. The computer-readable storage medium of claim 1 wherein said method further comprises:

rendering said one or more frames of said first selected camera feed as an animated texture on a billboard that is superimposed onto said map display of said multi-dimensional virtual map.

5. The computer-readable storage medium of claim 1 wherein:

said multi-dimensional virtual map comprises a digital elevation map (DEM), and

said method further comprises projecting said one or more frames of said first selected feed onto said DEM.

6. The computer-readable storage medium of claim 1 wherein said method further comprises:

removing backgrounds from said one or more frames of said first selected camera feed so that only moving objects are overlaid onto said map display of said multi-dimensional virtual map.

7. The computer-readable storage medium of claim 1 wherein said method further comprises:

replacing actual images of objects in said one or more frames of said first selected camera feed with simulations used in place of said actual images of said objects in said one or more frames of said first selected camera feed.

8. The computer-readable storage medium of claim 1 wherein said method further comprises:

correlating metadata associated with said map display of said multi-dimensional virtual map and information based on said first selected camera feed to calibrate a camera-associated with said first selected camera feed.

9. A computer system comprising:

a user interface operable for displaying information that identifies a plurality of available camera feeds and for receiving a user selection identifying a first selected camera feed and a second selected camera feed; and

memory storing computer-executable components including;

a virtual map model component operable for generating a map display of a multi-dimensional virtual map;

a video imaging component operable for receiving said first selected camera feed and said second selected camera feed; and

a visualization model component coupled to said virtual map model component and said video imaging component, wherein said virtualization model component is operable for;

accessing said first selected camera feed and said second selected camera feed from said video imaging component,

accessing data for rendering said map display of said multi-dimensional virtual map from said virtual map model component,

associating said first selected camera feed and said second selected camera feed with corresponding regions included in said map display of said multi-dimensional virtual map based on said data,

superimposing one or more frames of said first selected camera feed and one or more frames of said second selected camera feed onto said corresponding regions included in said map display of said multi-dimensional virtual map generated from said virtual map model, wherein said one or more frames of said first selected camera feed and said one or more frames of said second selected camera feed are aligned with respect to time, and

rendering said map display of said multi-dimensional virtual map with said one or more frames of said first selected camera feed and said one or more frames of said second selected camera feed being overlaid on said corresponding regions and with a remainder of said map display of said multi-dimensional virtual map being displayed around said corresponding regions.

10. The computer system of claim 9 wherein:

said first selected camera feed comprises a dynamic video feed of vehicles traveling on a roadway,

said map display of said multi-dimensional virtual map includes an image of said roadway, and

said visualization model component is operable for aligning said dynamic video feed with said image of said roadway.

11. The computer system of claim 10 wherein:

location and speed of said vehicles are tracked to compute positions of said vehicles with respect to time.

12. The computer system of claim 9 wherein said one or more frames of said first selected camera feed are rendered as an animated texture on a billboard that is superimposed onto said map display of said multi-dimensional virtual map.

13. The computer system of claim 9 wherein:

said multi-dimensional virtual map comprises a digital elevation map (DEM), and

said one or more frames of said first selected camera feed are projected onto said corresponding region in said DEM.

14. The computer system of claim 9 wherein backgrounds are removed from said one or more frames of said first selected camera feed so that only moving objects are overlaid onto said map display of said multi-dimensional virtual map.

15. The computer system of claim 9 wherein actual images of objects in said one or more frames of said first selected camera feed are replaced with simulations used in place of said actual images of said objects in said one or more frames of said first selected camera feed.

16. The computer system of claim 9 wherein said computer-executable components further include:

a calibration model component operable for calibrating a camera associated with said first selected camera feed using metadata associated with said map display of said multi-dimensional virtual map and information based on said first selected camera feed.

17. A computer-implemented method of visualizing camera feeds, said method comprising:

displaying, by a computing device, information that identifies a plurality of available camera feeds;

receiving, by said computing device, a user selection identifying a first selected camera feed and a second selected camera feed;

accessing, by said computing device, said first selected camera feed and said second selected camera feed from said plurality of available camera feeds;

accessing, by said computing device, data for rendering a map display of a multi-dimensional virtual map;

associating, by said computing device, said first selected camera feed and said second selected camera feed with corresponding regions included in said map display of said multi-dimensional virtual map based on said data;

superimposing, by said computing device, one or more frames of said first selected camera feed and one or more frames of said second selected camera feed onto said corresponding regions included in said map display of said multi-dimensional virtual map, wherein said one or more frames of said first selected camera feed and said one or more frames of said second selected camera feed are aligned with respect to time; and

rendering, by said computing device, said map display of said multi-dimensional virtual map with said one or more frames of said first selected camera feed and said one or more frames of said second selected camera feed being overlaid on said corresponding regions and with a remainder of said map display of said multi-dimensional virtual map being displayed around said corresponding regions.

18. The method of claim 17 wherein:

said first selected camera feed comprises a dynamic video feed of vehicles traveling on a roadway,

said map display of said multi-dimensional virtual map includes an image of said roadway, and

said method further comprises aligning said dynamic video feed with said image of said roadway.

19. The method of claim 18 further comprising:

tracking, by said computing device, location and speed of said vehicles to compute positions of said vehicles with respect to time.

20. The method of claim 17 further comprising:

correlating, by said computing device, metadata associated with said map display of said multi-dimensional virtual map and information based on said first selected camera feed to calibrate a camera associated with said first selected camera feed.