System and method for forming a video stream containing GIS data in real-time

• US 10,325,350 B2
• Filed: 06/08/2012
• Issued: 06/18/2019
• Est. Priority Date: 06/10/2011
• Status: Active Grant

• Current Assignee: Pictometry International Corporation (EagleView Technologies Incorporated)
• Original Assignee: Pictometry International Corporation (EagleView Technologies Incorporated)
• Inventors: Schultz, Stephen
• Primary Examiner(s): Rahaman, Mohammed S
• Assistant Examiner(s): Lee, Jimmy S
• Application Number:
  US13/492,107
• Publication Number:
  US 20120314068A1
• Time in Patent Office: 2,566 Days
• Field of Search: 348116, 348144, 348E07085
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images

Abstract

Image capture systems and methods may include one or more video capture devices capable of capturing one or more video frames. The video frames may include geographic position data and orientation data associated therewith, and may be stored on one or more non-transient computer readable medium. The computer system may marshal each video frame to one or more processors from a bank of processors for geo-referencing and overlaying of geographic information system (GIS) data on the video frames in real time.

Claims

1. A method, comprising the steps of:

capturing one or more video frames of a geographic area with one or more video capture devices, the video frames having pixels;

storing the video frames on one or more non-transient computer readable medium accessible by a computer system;

associating geographic position data and orientation data of the one or more video capture devices with each video frame;

marshaling, by the computer system, the video frames to one or more processors from a bank of processors of the computer system;

analyzing the geographic position data and orientation data and the video frames to generate geo-referencing data for pixels of the video frames with the bank of processors, wherein the geo-referencing data comprises geographic locations of pixels of the video frame;

determining a geographical boundary of the video frame from the geo-referencing data;

obtaining one or more layers of geographic information system (GIS) data using the determined geographical boundary of the video frame; and

determining with the bank of processors overlay position of the geographic information system (GIS) data on the video frames in real time based at least in part on the geo-referencing data.

2. The method of claim 1, wherein the one or more video frames includes one or more series of video frames.

3. The method of claim 1, wherein the one or more video frames is captured at a full-motion video frame rate.

4. The method of claim 1, wherein one or more of the video capture devices are secured to a moving platform.

5. The method of claim 4, wherein one or more of the video capture devices are mounted on a stationary platform.

6. The method of claim 1, wherein storing the video frames further includes storing interior geometry calibration data for the one or more video capture devices.

7. The method of claim 6, wherein geo-reference data for the video frames includes utilizing one or more of the geographic position data, the orientation data, and the interior geometry calibration data.

8. The method of claim 1, further comprising the steps of overlaying the GIS data on the video frames utilizing the overlay position of the GIS data to form composite video frames having overlaid GIS data, and storing the composite video frames in real-time.

9. The method of claim 8, further comprising the steps of assembling the video frames into a full motion video series and enabling display of the full motion video series in real-time.

10. The method of claim 8, wherein the composite video frames are stored over the video frames.

11. The method of claim 8, wherein the composite video frames are stored separately from the video frames.

12. The method of claim 1, further comprising the steps of overlaying the GIS data on the video frames utilizing the overlay position of the GIS data to form composite video frames having overlaid GIS data and displaying one or more composite video frames in non-real-time and one or more composite video frames in real-time.

13. The method of claim 1, further comprising the steps of overlaying the GIS data on the video frames utilizing the overlay position of the GIS data to form composite video frames having overlaid GIS data and providing the composite video frames via a website in real-time.

14. The method of claim 1, wherein the overlay position includes one or more video frame pixel row and one or more video frame pixel column.

15. A method, comprising the steps of:

capturing a series of video frames of a geographic area with one or more cameras from a moving platform and storing the video frames on one or more non-transitory computer readable medium accessible by a computer system while also recording geographic position data and orientation data of the one or more camera on the one or more non-transitory computer readable medium, and with interior geometry calibration data for the one or more cameras being stored on the one or more non-transitory computer readable medium, the video frames having pixels;

performing in real-time the following steps by the computer system for the video frames;

associating the geographic position data and orientation data with the video frames;

analyzing pixels of the video frames to generate geo-referencing data for one or more video frames utilizing the geographic position data, the orientation data and the interior geometry calibration data, wherein the geo-referencing data comprises geographic locations of pixels of the video frame;

determining geographical boundaries of the video frames from the geo-referencing data;

accessing geographic information system (GIS) data using the determined geographical boundary of the video frames; and

determining an overlay position of the GIS data relative to the video frames for which geo-referencing data is generated in real-time based at least in part on the generated geo-reference data.

16. The method of claim 15, further comprising the steps of overlaying the GIS data on the video frames utilizing the overlay position of the GIS data to form composite video frames having overlaid GIS data and storing the composite video frames in real-time.

17. The method of claim 16, further comprising the steps of assembling the video frames into a full motion video series and enabling display of the full motion video series in real-time.

18. The method of claim 16, wherein the composite video frames are stored over the video frames.

19. The method of claim 16, wherein the composite video frames are stored separately from the video frames.

20. The method of claim 15, wherein the step of generating geo-referencing data includes marshaling video frames to multiple processors in a predetermined sequence.

21. The method of claim 20, wherein the predetermined sequence is a round-robin sequence.

22. The method of claim 15, wherein the step of geo-referencing data includes the step of calculating an offset of a first video frame relative to a second video frame, and using the offset to generate the geo-referencing data for at least one of the first and second video frames.

23. An image capture system comprising:

one or more cameras adapted to be mounted to a moving platform, the one or more cameras adapted to capture a series of video frames of a geographic area, the video frames having pixels;

one or more position and orientation system adapted to record geographic position data and orientation data; and

a computer system having one or more processors and one or more non-transitory memory storing processor executable code and interior geometry calibration data for the one or more cameras, and communicating with the one or more processor;

wherein the processor executable code, when executed by the one or more processors causes the one or more processors to (1) receive a series of video frames from the one or more cameras, (2) store the video frames in the one or more non-transitory memory, (3) record geographic position data and orientation data of the one or more camera on the one or more non-transitory memory, (4) associate the geographic position data and orientation data with the video frames, (5) analyze pixels of the video frames to generate geo-referencing data for each video frame utilizing the geographic position data, the orientation data, and the interior geometry calibration data, wherein the geo-referencing data comprises geographic locations of pixels of the video frame;

(6) determine a geographical boundary of the video frame from the geo-referencing data, (7) access GIS data from a GIS database using the determined geographical boundary of the video frame, and (8) determine an overlay position of the GIS data relative to the video frames based at least in part on the geo-referencing data.

24. A computer system comprising:

one or more processors; and

one or more non-transitory memory storing processor executable code and interior geometry calibration data for one or more cameras, the one or more non-transitory memory communicating with the one or more processor;

wherein the processor executable code, when executed by the one or more processors, causes the one or more processors to receive full motion video having at least one series of video frames from at least one camera, the video frames having pixels;

analyze pixels of the video frames to generate geo-referencing data for certain video frames within the series of video frames, wherein the geo-referencing data comprises geographic locations of pixels of the video frame;

determine a geographical boundary of the video frame from the geo-referencing data;

obtain one or more layers of geographic information system (GIS) data using the determined geographical boundary of the video frame; and

overlay the GIS data onto video frames utilizing the geo-referencing data to form a series of composite video frames in real-time such that the GIS data can be perceived by an individual viewing a series of the composite video frames provided at a full motion video rate.

25. The computer system of claim 24, wherein the processor executable code, when executed by the one or more processors causes the one or more processors to record geographic position data and orientation data of the one or more camera on the one or more non-transitory memory, associate the geographic position data and orientation data with the video frames, generate the geo-referencing data for each video frame utilizing the geographic position data, the orientation data, and the interior geometry calibration data, access GIS data from a GIS database using the determined geographical boundary of the video frame, and determine an overlay position of the GIS data relative to the video frames based at least in part on the geo-referencing data.