System and method for video image registration in a heads up display

• US 20060121993A1
• Filed: 12/02/2004
• Published: 06/08/2006
• Est. Priority Date: 12/02/2004
• Status: Active Grant

• Current Assignee: Science Applications International Corporation
• Original Assignee: Science Applications International Corporation
• Inventors: Scales, John Richard, Hose, Mark David
• Granted Patent: US 7,787,012 B2
• Time in Patent Office: Days
• US Class Current: 463/48
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/245 specially adapted to a particular type of game, e.g. steering wheels
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 13/837 Shooting of targets
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1062 being specially adapted to a type of game, e.g. steering wheel
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/8076 Shooting
• CPC Class Codes: F41G 3/16 Sighting devices adapted for indirect laying of fire

Abstract

A system and method for aligning video images with an underlying visual field are provided. A video camera is coupled with a heads up display, and a computer positions images from the video camera on the heads up display based on the relative orientations of the camera and the display. As the video camera moves with respect to the display, the images are repositioned within the heads up display. The video image, which may, for example, come from a weapon sight, is aligned within the heads up display so that an observer can easily view the camera image without having to shift focus from the larger scene.

Claims

1. A method of registering video images with an underlying visual field comprising the steps of:

(1) determining a source orientation of a video source of a video feed;

(2) determining a display orientation of a transparent display overlaying the visual field;

(3) displaying a portion of the video feed in the transparent display; and

(4) registering the portion of the video feed with the underlying visual field.

2. The method of claim 1, wherein step (3) comprises displaying the portion of the video feed in a heads up display (HUD).

3. The method of claim 2, wherein the HUD is housed in a pair of night-vision goggles.

4. The method of claim 1, further comprising:

(5) cropping the portion of the video feed such that less than an entire image is displayed.

5. The method of claim 1, further comprising:

(5) resizing the portion of the video feed such that the video feed is magnified or compressed.

6. The method of claim 1, wherein the video source is attached to a weapon.

7. The method of claim 5, wherein the video source is a thermal gun sight.

8. The method of claim 1, wherein step (1) comprises sensing at least one of a pitch value, a yaw value, and a roll value for the video source.

9. The method of claim 1, wherein step (2) comprises sensing at least one of a pitch value, a yaw value, and a roll value for the transparent display.

10. The method of claim 1, wherein step (1) comprises determining a source orientation using data derived from a plurality of radio devices attached to the video source.

11. The method of claim 1, wherein step (2) comprises determining a display orientation using data derived from a plurality of radio devices attached to the transparent display.

12. The method of claim 1, further comprising:

(5) repositioning the portion of the video feed within the transparent display when the video source or transparent display moves.

13. The method of claim 1, further comprising the steps of:

(5) determining a vertical position for the portion of the video feed by calculating a difference between a pitch value of the video source and a pitch value of the transparent display; and

(6) determining a horizontal position for the portion of the video feed by calculating a difference between a yaw value of the video source and a yaw value of the transparent display.

14. The method of claim 13, further comprising the steps of:

(7) determining a rotational orientation for the portion of the video feed within the visual field by calculating a difference between a roll value of the video source and a roll value of the observer.

15. A system for displaying at least a portion of a video feed overlaying a visual field of an observer comprising:

a video camera providing the video feed;

a first orientation sensor that detects the orientation of the video camera;

a heads up display (HUD);

a second orientation sensor that detects the orientation of the HUD; and

a computer adapted to receive sensor data from the first and second orientation sensors, to receive the video feed from the video camera, to display the at least a portion of a video feed in the HUD, and to align the at least a portion of the video feed with the visual field of the observer.

16. The system of claim 15, wherein the computer is further adapted to calculate a position for the at least a portion of the video feed based on pitch and yaw differences between the video camera and the HUD.

17. The system of claim 15, wherein the computer is further adapted to calculate a rotational orientation for the at least a portion of the video feed within the visual field.

18. The system of claim 17, wherein the computer is further adapted to calculate the rotational orientation for the at least a portion of the video feed based on a difference in roll values between the video camera and the HUD.

19. The system of claim 15, wherein the computer is further adapted to receive sensor data from the first and second orientation sensors using a wireless radio.

20. The system of claim 15, wherein the HUD is housed in a pair of goggles.

21. The system of claim 15, wherein the HUD is mounted in a vehicle.

22. The system of claim 15, wherein the video camera is attached to a weapon and the video feed is of a line of sight of the weapon.

23. The system of claim 22, wherein the video camera produces a magnified thermal image.

24. The system of claim 15, wherein the first orientation sensor and the second orientation sensor use the earth'"'"'s gravitational and magnetic fields to detect orientation.

25. The system of claim 15, further comprising:

a rangefinder.

26. The system of claim 25, wherein the computer is further adapted to utilize data from the rangefinder for aligning the at least a portion of the video feed with the visual field of the observer.

27. A method of registering a first video image with a second video image comprising:

(1) determining a first source orientation for a source of the first video image;

(2) determining a second source orientation for a source of the second video image;

(3) displaying at least a portion of the first video image over the second video image; and

(4) aligning the at least a portion of the first video image with the second video image using the first and second source orientations.

28. The method of claim 27, wherein step (4) further comprises positioning and orienting the first video image using image registration techniques.

29. A system for registering a video image with a visual field comprising:

a weapon sight attached to a weapon, the weapon sight adapted to provide the video image;

a first orientation sensor affixed to the weapon;

a heads up display, through which the visual field is viewable;

a second orientation sensor affixed to the heads up display;

a computer adapted to perform the steps of;

receiving the video image from the weapon sight;

resizing the video image;

cropping the video image;

transmitting the video image to the heads up display; and

aligning the video image with the visual field using data from the first and second orientation sensors, such that the video image is registered with the visual field.