SYSTEM AND METHOD FOR REAL-TIME OVERLAY OF MAP FEATURES ONTO A VIDEO FEED
First Claim
1. A method for providing an augmented video feed obtained by a camera of a manned or unmanned aerial vehicle (“
- UAV”
) to a user interface (“
UI”
), the method comprising;
obtaining a sequence of video images with or without corresponding sensor metadata from the aerial vehicle;
obtaining supplemental data based on the sequence of video images and the sensor metadata;
correcting, by a processor, an error in the sensor metadata using a reconstruction error minimization technique;
creating, by a processor, a geographically-referenced scene model based on a virtual sensor coordinate system that is registered to the sequence of video images;
overlaying the supplemental information onto the geographically-referenced scene model by rendering geo-registered data from a 3D perspective that matches a corrected camera model;
creating a video stream of a virtual representation from the scene from the perspective of the camera based on the overlaying; and
providing the video stream to a UI to be render onto a display.
4 Assignments
0 Petitions
Accused Products
Abstract
A method is provided for augmenting video feed obtained by a camera of a aerial vehicle to a user interface. The method can include obtaining a sequence of video images with or without corresponding sensor metadata from the aerial vehicle; obtaining supplemental data based on the sequence of video images and the sensor metadata; correcting an error in the sensor metadata using a reconstruction error minimization technique; creating a geographically-referenced scene model based on a virtual sensor coordinate system that is registered to the sequence of video images; overlaying the supplemental information onto the geographically-referenced scene model by rendering geo-registered data from a 3D perspective that matches a corrected camera model; creating a video stream of a virtual representation from the scene from the perspective of the camera based on the overlaying; and providing the video stream to a UI to be render onto a display.
-
Citations
20 Claims
-
1. A method for providing an augmented video feed obtained by a camera of a manned or unmanned aerial vehicle (“
- UAV”
) to a user interface (“
UI”
), the method comprising;obtaining a sequence of video images with or without corresponding sensor metadata from the aerial vehicle; obtaining supplemental data based on the sequence of video images and the sensor metadata; correcting, by a processor, an error in the sensor metadata using a reconstruction error minimization technique; creating, by a processor, a geographically-referenced scene model based on a virtual sensor coordinate system that is registered to the sequence of video images; overlaying the supplemental information onto the geographically-referenced scene model by rendering geo-registered data from a 3D perspective that matches a corrected camera model; creating a video stream of a virtual representation from the scene from the perspective of the camera based on the overlaying; and providing the video stream to a UI to be render onto a display. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
- UAV”
-
13. A device for providing an augmented video feed obtained by a camera of a manned or unmanned aerial vehicle (“
- UAV”
) to a user interface (“
UI”
), the device comprising;a memory containing instructions; and at least one processor, operably connected to the memory, the executes the instructions to perform a method for providing an augmented video feed obtained by a camera of a manned or unmanned aerial vehicle (“
UAV”
) to a user interface (“
UI”
), comprising;obtaining a sequence of video images with or without corresponding sensor metadata from the aerial vehicle; obtaining supplemental data based on the sequence of video images and the sensor metadata; correcting, by a processor, an error in the sensor metadata using a reconstruction error minimization technique; creating, by a processor, a geographically-referenced scene model based on a virtual sensor coordinate system that is registered to the sequence of video images; overlaying the supplemental information onto the geographically-referenced scene model by rendering geo-registered data from a 3D perspective that matches a corrected camera model; creating a video stream of a virtual representation from the scene from the perspective of the camera based on the overlaying; and providing the video stream to a UI to be render onto a display. - View Dependent Claims (14, 15, 16, 17)
- UAV”
-
18. A computer readable storage medium comprising instructions for causing one or more processors to perform a method, the method for discerning a vehicle at an access control point, the device comprising:
-
obtaining a sequence of video images with or without corresponding sensor metadata from the aerial vehicle; obtaining supplemental data based on the sequence of video images and the sensor metadata; correcting, by a processor, an error in the sensor metadata using a reconstruction error minimization technique; creating, by a processor, a geographically-referenced scene model based on a virtual sensor coordinate system that is registered to the sequence of video images; overlaying the supplemental information onto the geographically-referenced scene model by rendering geo-registered data from a 3D perspective that matches a corrected camera model; creating a video stream of a virtual representation from the scene from the perspective of the camera based on the overlaying; and providing the video stream to a UI to be render onto a display. - View Dependent Claims (19, 20)
-
Specification