Automatic video generation for navigation and object finding

• US 8,838,381 B1
• Filed: 11/10/2009
• Issued: 09/16/2014
• Est. Priority Date: 11/10/2009
• Status: Active Grant

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Daily, Michael, Azuma, Ronald
• Primary Examiner(s): Black, Thomas
• Assistant Examiner(s): Nolan, Peter D
• Application Number:
  US12/590,575
• Time in Patent Office: 1,771 Days
• Field of Search: 701/428, 701/431, 701/433, 701/436, 701/454, 701/516, 345/629, 345/632, 345/633, 345/638
• US Class Current: 701/436
• CPC Class Codes: G01C 21/3647 Guidance involving output of stored or live camera images or video streams
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images

Abstract

Described is a system for conveying the spatial location of an object with respect to a user'"'"'s current location utilizing a video rendering following an automatically generated path from the user'"'"'s location to the location of the object of interest initiated from the user'"'"'s current perspective. The system comprises a display device; a virtual three-dimensional model of the user'"'"'s environment; a visualization creation module; a route planner; and a video rendering following an automatically generated path from the user'"'"'s location to the location of the object of interest utilizing the visualization creation module, wherein the video rendering displayed on the display device is from a first-person view. Also described is a method of utilizing the system.

Claims

1. A system for conveying the spatial location of an object with respect to a user'"'"'s current location, the system comprising:

a display device;

at least one sensor for computing the user'"'"'s current position and orientation relative to an object of interest;

a virtual three-dimensional model of the user'"'"'s environment generated from location information detected in six dimensions comprising x, y, z, roll, pitch, and yaw dimensions;

a route planner configured to generate a route given a start location and an end location, wherein the start location is the user'"'"'s current position and orientation and the end location is the location of the object of interest; and

a visualization creation module configured to create a video rendering following the route created by the route planner by integrating the virtual three-dimensional model of the user'"'"'s environment and an image of the user'"'"'s current environment, wherein the video rendering begins with the image of the user'"'"'s current environment and seamlessly transitions to a sequence of frames comprising the virtual three-dimensional model of the user'"'"'s environment, wherein the video rendering is displayed on the display device starting from the user'"'"'s current viewpoint as seen in the image of the user'"'"'s current environment, and wherein the visualization creation module utilizes faster than real-time rendering of the sequences of frames for the video rendering.

2. The system for conveying the spatial location of an object with respect to a user'"'"'s current location as set forth in claim 1, wherein the display device comprises sensors for estimating the user'"'"'s position and orientation.

3. The system for conveying the spatial location of an object with respect to a user'"'"'s current location as set forth in claim 2, wherein the sensors are selected from a group consisting of tilt and accelerometer sensors.

4. The system for conveying the spatial location of an object with respect to a user'"'"'s current location as set forth in claim 1, wherein the display device further comprises a camera configured to capture images from the user'"'"'s current viewpoint, such that the images provide information regarding the user'"'"'s current position and orientation.

5. The system for conveying the spatial location of an object with respect to a user'"'"'s current location as set forth in claim 1, the system further comprising a remote server for communication with the display device.

6. A method for conveying, the spatial location of an object with respect to a user'"'"'s current location, comprising the steps of:

computing the user'"'"'s current position and orientation relative to an object of interest;

creating a virtual three-dimensional model of the user'"'"'s environment using location information detected in six dimensions comprising x, y, z, roll, pitch, and yaw dimensions;

generating a route given a start location and an end location, wherein the start location is the user'"'"'s current position and orientation and the end location is the location of the object of interest; and

creating a video rendering following the generated route by integrating the virtual three-dimensional model of the user'"'"'s environment and an image of the user'"'"'s current environment, wherein the video rendering begins with the image of the user'"'"'s current environment and seamlessly transitions to a sequence of frames comprising the virtual three-dimensional model of the user'"'"'s environment, wherein the video rendering is displayed on a display device starting from the user'"'"'s current viewpoint as seen in the image of the user'"'"'s current environment, and wherein the visualization creation module utilizes faster than real-time rendering of the sequences of frames for the video rendering.

7. The method for conveying, the spatial location of an object with respect to a user'"'"'s current location as set forth in claim 6, further comprising transmitting, information of the user'"'"'s position and orientation through a wireless network to a remote server, wherein the computation of the user'"'"'s orientation is performed on the remote server.

8. The method for conveying the spatial location of an object with respect to a user'"'"'s current location as set forth in claim 7, further comprising transmitting the video rendering from the remote server to the user through the wireless network.

9. The method for conveying the spatial location of an object with respect to a user'"'"'s current location as set forth in claim 6, further comprising capturing images with a camera in the display device from the user'"'"'s current viewpoint, such that the images provide information regarding the user'"'"'s current position and orientation.

10. The method for conveying the spatial location of an object with respect to a user'"'"'s current location as set forth in claim 6, wherein the user'"'"'s position and orientation are computed using sensors in the display device.

11. The method for conveying the spatial location of an object with respect to a user'"'"'s current location as set forth in claim 10, wherein the sensors are selected from a group consisting of tilt and accelerometer sensors.

12. A computer program product for conveying the spatial location of an object with respect to a user'"'"'s current location, the computer program product comprising computer-readable instruction means stored on a non-transitory computer-readable medium that are executable by a computer for causing the computer to perform operations of:

computing the user'"'"'s current position and orientation relative to an object of interest;

creating a virtual three-dimensional model of the user'"'"'s environment using location information detected in six dimensions comprising x, y, z, roll, pitch, and yaw dimensions;

generating a route given a start location and an end location, wherein the start location is the user'"'"'s current position and orientation and the end location is the location of the object of interest; and

creating a video rendering following the generated route by integrating the virtual three-dimensional model of the user'"'"'s environment and an image of the user'"'"'s current environment, wherein the video rendering begins with the image of the user'"'"'s current environment and seamlessly transitions to a sequence of frames comprising the virtual three-dimensional model of the user'"'"'s environment, wherein the video rendering is displayed on a display device starting from the user'"'"'s current viewpoint as seen in the image of the user'"'"'s current environment, and wherein the visualization creation module utilizes faster than real-time rendering of the sequences of frames for the video rendering.