Birds eye view virtual imaging for real time composited wide field of view

• US 8,295,644 B2
• Filed: 10/27/2008
• Issued: 10/23/2012
• Est. Priority Date: 08/09/2003
• Status: Active Grant

• Current Assignee: William E. Knight
• Original Assignee: Doubleshot, Inc.
• Inventors: Shulman, Alan, Snyder, III, Donald R.
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US12/259,227
• Publication Number:
  US 20090092334A1
• Time in Patent Office: 1,457 Days
• Field of Search: 382/100, 382/104, 382/154, 382293-295, 382/302, 382/298, 348/42, 348/47
• US Class Current: 382/298
• CPC Class Codes: G01C 21/3647 Guidance involving output of stored or live camera images or video streams
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

A live image and a previously acquired or generated image are superimposed or composited to represented a virtual vantage point for flying, driving or navigating a plane, vehicle or vessel.

Claims

1. A method for displaying a composite image of alternative vantage points, comprising the steps of:

acquiring a first image from an image acquisition device;

acquiring a second image from the image acquisition device;

generating a first vantage point image and a second vantage point image by transforming the first image and the second image, respectively,

wherein transforming the first and second images into the first and second vantage point images comprises transforming each row of the x axis gradually with increased compression starting from the upper edge of the frame with increasing compression towards the bottom of the frame;

scaling the second vantage point image based on at least one of a scaling factor, an x axis, and a y axis displacement; and

creating a composite image by overlaying the scaled second vantage point image onto the first vantage point image.

2. The method of claim 1, further comprising acquiring the first image from an image database.

3. The method of claim 1, wherein the second image is a real time image captured after the first image following movement or displacement of the image acquisition device.

4. The method of claim 1, wherein transforming the first image and the second image comprises generating a bird'"'"'s-eye view of the first image and the second image using techniques comprising image compression, converting forward-looking video or motion picture imagery into a down-looking image database, or geo-spatial registration.

5. The method of claim 1, wherein scaling the second vantage point image is performed based in part on a trapezoidal shape.

6. A method for displaying a composite image of alternative vantage points viewed from a vehicle, comprising the steps of:

acquiring a first image from an image acquisition device associated with a vehicle;

acquiring a second image from the image acquisition device, wherein the second image is captured after the first image following movement or displacement of the image acquisition device;

generating a first vantage point image and a second vantage point image by transforming the first image and the second image, respectively,

wherein transforming the first and second images into the first and second vantage point images comprises transforming each row of the x axis gradually with increased compression starting from the upper edge of the frame with increasing compression towards the bottom of the frame;

scaling the second vantage point image based on at least one of a scaling factor, an x axis, a y axis, and a z axis displacement of the vehicle; and

creating a composite image by overlaying the scaled second vantage point image onto the first vantage point image.

7. The method of claim 6, further comprising acquiring a first image from an image database.

8. The method of claim 6, wherein transforming the first image and the second image comprises generating a bird'"'"'s-eye view of the first image and the second image using techniques comprising image compression, converting forward-looking video or motion picture imagery into a down-looking image database, or geo-spatial registration.

9. The method of claim 6, further comprising generating the scaled second vantage point image based on a de-magnification factor, wherein the de-magnification factor is based on the y axis displacement of the vehicle between the acquisition of the first image and the second image.

10. The method of claim 6, wherein the composite image includes a synthesized image of the vehicle.

11. The method of claim 6, wherein the composite image comprises various indicia, including road markings, road hazards, and intended driving route.

12. The method of claim 6, wherein the composite image is refreshed in real time based on the forward progress of the vehicle.

13. The method of claim 6, wherein creating the composite image by overlaying the scaled second vantage point image onto the first vantage point image comprises the steps of:

merging the first vantage point image with the scaled second vantage point image based on a lateral translation and a rotation of the image acquisition device between acquisition of the first image and the second image; and

mapping a displacement of pixels from the first vantage point image to the scaled second vantage point image by replacing a sub group of pixels associated with the first vantage point image with the scaled second vantage point image.

14. A method for displaying a composite image of alternative vantage points viewed from a plurality of vehicles, comprising the steps of:

acquiring a plurality of first images from a plurality of image acquisition devices associated with each of a plurality of vehicles;

acquiring a plurality of second images from the plurality of image acquisition devices, wherein the plurality of second images are captured after the plurality of first images;

generating a plurality of first vantage point images and a plurality of second vantage point images by transforming the plurality of first images and the plurality of second images, respectively,

wherein transforming the first and second images into the first and second vantage point images comprises transforming each row of the x axis gradually with increased compression starting from the upper edge of the frame with increasing compression towards the bottom of the frame;

scaling the plurality of second vantage point images based on at least one of a scaling factor, an x axis, a y axis, and a z axis displacement of the vehicle; and

creating a composite display by overlaying the plurality of scaled second vantage point images onto the plurality of first vantage point images, wherein the composite display is accessible from each of the plurality of vehicles.

15. A system for displaying a composite image of alternative vantage points comprising:

an image acquisition device for acquiring a first image and a second image;

an image processor for performing the steps comprising;

generating a first vantage point image and a second vantage point image by transforming the first image and the second image, respectively,

wherein transforming the first and second images into the first and second vantage point images comprises transforming each row of the x axis gradually with increased compression starting from the upper edge of the frame with increasing compression towards the bottom of the frame;

scaling the second vantage point image based on at least one of a scaling factor, an x axis, and a y axis displacement; and

creating a composite image by overlaying the scaled second vantage point image onto the first vantage point image.

16. The system of claim 15, further comprising an image database for generating a first image.

17. The system of claim 15, wherein the second image is a real time image captured after the first image following movement or displacement of the image capturing device.

18. A system for displaying a composite image of alternative vantage points viewed from a plurality of vehicles comprising:

a plurality of image acquisition devices associated with each of a plurality of vehicles for acquiring a plurality of first images and a plurality of second images, wherein the plurality of second images are captured after the plurality of first images; and

an image processor for performing the steps comprising;

generating a plurality of first vantage point images and a plurality of second vantage point images by transforming the plurality of first images and the plurality of second images, respectively,

wherein transforming the first and second images into the first and second vantage point images comprises transforming each row of the x axis gradually with increased compression starting from the upper edge of the frame with increasing compression towards the bottom of the frame;

scaling the plurality of second vantage point images based on at least one of a scaling factor, an x axis, a y axis, and a z axis displacement of the vehicle; and

creating a composite image by overlaying the plurality of scaled second vantage point images onto the plurality of first vantage point images, wherein the composite image is accessible from each the plurality of vehicles.