Video with map overlay

• US 7,456,847 B2
• Filed: 08/12/2004
• Issued: 11/25/2008
• Est. Priority Date: 08/12/2004
• Status: Expired due to Fees

• Current Assignee: Russell Steven Krajec
• Original Assignee: Russell Steven Krajec
• Inventors: Krajec, Russell Steven
• Primary Examiner(s): Nguyen; Phu K
• Application Number:
  US10/710,904
• Publication Number:
  US 20040239688A1
• Time in Patent Office: 1,566 Days
• Field of Search: 345/629, 345/419, 345/581, 345/157, 345/158, 340/901, 340/908, 340/989, 340/990, 701/207, 701/208, 701/216, 701/217
• US Class Current: 345/629
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: H04N 5/265 Mixing

Abstract

A system and method for placing reference locators on a streaming video image taken from a camera mounted on a vehicle. Sensors monitoring the camera field of view and direction of the optical axis along with sensors monitoring the vehicle'"'"'s position are used to generate a video overlay having various indicia. The calculated positions of the indicia may be compared to the position of landmarks taken from the video image to compensate for any inaccuracies in the calculated position taken from the various sensors.

Claims

1. A system comprising:

a vehicle having vehicle position sensors;

a first video camera mounted to said vehicle and producing a first video feed in a first camera field of view;

said first video camera having a first optical axis;

a second video camera having camera position sensors, a second optical axis, and a second camera field of view, said second video camera being mounted to said vehicle, said second camera field of view being larger than said first camera field of view, said second video camera further producing a second video feed;

a field of view calculator adapted to;

use said vehicle position sensors and said camera position sensors to produce a first calculated field of view;

determine that a first landmark is approximately located within said first camera field of view;

determine a calculated position of said first landmark within said second camera field of view;

find the actual position of said landmark within said second video feed;

compare said calculated position and said actual position of said landmark to determine a set of correction factors; and

use said correction factors to change said first calculated field of view;

an indicia generator adapted to use said first calculated field of view to produce at least one indicator within said first camera field of view;

a video display adapted to display said first video feed and said indicator.

2. The system of claim 1 wherein said first optical axis is substantially parallel to said first optical axis.

3. The system of claim 1 wherein said first optical axis is not parallel to said first optical axis.

4. The system of claim 1 wherein said landmark comprises a stationary object.

5. The system of claim 1 wherein said landmark comprises a movable object.

6. The system of claim 5 wherein said landmark comprises a second vehicle having a set of second vehicle sensors in communication with said field of view calculator.

7. The system of claim 1 wherein said vehicle is one of a group comprising an aircraft, a land vehicle, and a watercraft.

8. The system of claim 1 wherein said vehicle position sensors comprises at least one from a group comprising a global positioning system receiver, an altimeter, a compass, and a gyroscope.

9. The system of claim 1 wherein said camera position sensors comprises at least one from a group comprising a roll sensor, a pitch sensor, a yaw sensor, and a zoom sensor.

10. The system of claim 1 wherein said indicia comprises at least one from a group comprising latitude and longitude coordinates, a street name, a building address, a building name, and a vehicle.

11. A method comprising:

generating a first video stream from a vehicle, said first video stream having a first optical axis;

determining the approximate position of said vehicle;

determining the optical axis of said first video stream and the first field of view;

determining a first calculated field of view from said position of said vehicle and said first field of view;

calculating the position of an indicator using said first calculated field of view;

determining that a first landmark is approximately located within the first field of view;

determining a calculated position of said first landmark within said first field of view;

finding the actual position of said landmark within said first video feed;

comparing said calculated position and said actual position of said landmark to determine a set of correction factors;

using said correction factors to change said first calculated field of view;

generating a second video stream from said vehicle, said second video stream having a second optical axis and a second field of view, said second field of view being smaller than said first field of view;

determining the position of said second optical axis and said second field of view;

generating an image of said indicator; and

displaying said image of said indicator with said second video stream.

12. The method of claim 11 wherein said first optical axis and said second optical axis are substantially parallel.

13. The method of claim 11 wherein said first optical axis and said second optical axis are not substantially parallel.

14. The method of claim 11 wherein said landmark comprises a stationary object.

15. The method of claim 11 wherein said landmark comprises a movable object.

16. The method of claim 15 wherein said landmark comprises a second vehicle having a set of second vehicle sensors in communication with said field of view calculator.

17. The method of claim 11 wherein said vehicle is one from a group comprising an aircraft, a land vehicle, and a watercraft.

18. The method of claim 11 wherein said determining the position of said vehicle comprises at least one output from a group comprising a global positioning system receiver, an altimeter, a compass, and a gyroscope.

19. The method of claim 11 wherein said determining the optical axis of said first video stream and the first field of view comprises at least one output from a group comprising a roll sensor, a pitch sensor, a yaw sensor, and a zoom sensor.

20. The method of claim 11 wherein said indicator comprises at least one from a group comprising latitude and longitude coordinates, a street name, a building address, a building name, and a vehicle.