Waveguide based fused vision system for a helmet mounted or head worn application

• US 10,642,038 B1
• Filed: 01/30/2017
• Issued: 05/05/2020
• Est. Priority Date: 01/30/2017
• Status: Active Grant

• Current Assignee: Rockwell Collins, Inc. (Raytheon Technologies Corporation d/b/a RTX)
• Original Assignee: Rockwell Collins, Inc. (Raytheon Technologies Corporation d/b/a RTX)
• Inventors: Schuyler, William J., Murdock, Theresa M., Hendrick, Wyatt L., Civilikas, Robert G., O'Brien, Tyrone D.
• Primary Examiner(s): Rakowski, Cara E
• Assistant Examiner(s): Cruz, Magda
• Application Number:
  US15/420,029
• Time in Patent Office: 1,191 Days
• CPC Class Codes: A42B 1/247 Means for attaching eyewear arrangements for supporting by headgear G02C3/02
• CPC Class Codes: A42B 3/00 Helmets; Helmet covers ; Other protective head coverings hoods for welders A61F9/06; defence protection helmets F41H1/04
• CPC Class Codes: A42B 3/003 Helmet covers
• CPC Class Codes: A42B 3/04 Parts, details or accessories of helmets
• CPC Class Codes: A42B 3/0406 Accessories for helmets
• CPC Class Codes: A42B 3/042 Optical devices night vision goggles G02B23/12
• CPC Class Codes: F41G 3/225 Helmet sighting systems
• CPC Class Codes: F41G 7/2226 comparing the observed data with stored target data, e.g. target configuration data
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0154 with movable elements
• CPC Class Codes: G02B 2027/0156 with optionally usable elements
• CPC Class Codes: G02B 2027/0159 with mechanical means other than scaning means for positioning the whole image
• CPC Class Codes: G02B 2027/0161 characterised by the relative positioning of the constitutive elements
• CPC Class Codes: G02B 23/00 Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
• CPC Class Codes: G02B 23/12 with means for image conversion or intensification
• CPC Class Codes: G02B 23/125 head-mounted
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0149 characterised by mechanical features G02B27/0176 takes precedence
• CPC Class Codes: G02B 27/017 : Head mounted
• CPC Class Codes: G02B 27/0172 : characterised by optical features
• CPC Class Codes: G02B 27/10 : Beam splitting or combining systems polarising systems G02B27/28; mixing and splitting light signals using optical waveguides G02B6/28; beam shaping, e.g. changing the cross-sectional area, by dividing or superposing multiple light beams G02B27/0905
• CPC Class Codes: G02B 27/4205 : having a diffractive optical element [DOE] contributing to image formation, e.g. whereby modulation transfer function MTF or optical aberrations are relevant
• CPC Class Codes: G02B 5/18 : Diffraction gratings holographic optical elements G02B5/32, G03H; integrally combined with optical fibres G02B6/02057; for coupling light guides G02B6/34; integrally combined with optical integrated light guides G02B6/12; grating systems G02B27/44
• CPC Class Codes: G06F 1/163 : Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 : Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

A waveguide based fused vision system includes a helmet mounted or head worn sensor module and a display assembly. The display assembly includes a frame including a pair of waveguide combiners and a pair of projectors associated with waveguide combiners. The display assembly also includes a sensor module. The pair of projectors is disposed on opposite sides of a frame. The sensor module is disposed substantially above the frame and includes a first sensor, a second sensor, a video processing circuit, and a symbol generator. The video processing circuit is configured to merge first sensor information from the first sensor and first symbols from the symbol generator and to merge second symbols and the second sensor information for displaying conformally the real world scene.

Claims

1. A waveguide based fused vision system for a helmet or head worn application, comprising:

a first waveguide disposed in a frame;

a second waveguide disposed in the frame;

a first projector for the first waveguide, the first projector being configured to provide a first image to the first waveguide at a first outside peripheral portion of the frame, the first image and a real world scene being viewable by a first eye on the first waveguide;

a second projector for the second waveguide, the second projector being configured to provide a second image to the second waveguide at a second outside peripheral portion of the frame opposite the first outside peripheral portion, the second image and the real world scene being viewable by a second eye on the second waveguide; and

a sensor module disposed substantially above the frame comprising at least one first sensor directly above the first waveguide, a second sensor directly above the second waveguide, a video processing circuit, and a symbol generator, the video processing circuit being configured to merge first sensor information from the first sensor and first symbols from the symbol generator for the first image, wherein the first symbols and the first sensor information are provided conformally with the real world scene.

2. The system of claim 1, wherein the sensor module is attached to a rail, the rail being fixed to the helmet, the sensor module being slideably engaged to the rail.

3. The system of claim 2, wherein a bottom edge of the first waveguide extends above a bottom rim of the helmet when the sensor module is slid to a stowed position.

4. The system of claim 1, wherein the sensor module further comprises:

the second sensor being configured to be disposed above the second eye of a user, wherein the first sensor is configured to be disposed above the first eye of the user, the video processing circuit being configured to merge second sensor information from the second sensor and second symbols from the symbol generator for the second image, wherein the second symbols and the second sensor information are provided conformally with the real world scene; and

a third sensor disposed between the first sensor and the second sensor, wherein the video processing circuit is configured to merge third sensor information from the third sensor into the first image and the second image.

5. The system of claim 4, wherein the first and second sensor are low light or short wave infrared sensors and the third sensor is a thermal or a long wave infrared sensor.

6. The system of claim 4, wherein the first image and the second image form a three dimensional image when viewed on the first waveguide and the second waveguide.

7. The system of claim 6, wherein the video processing circuit is configured to provide contrast, white balance, non-uniformity correction, histogram correction and brightness adjustments for the first and second image.

8. The system of claim 7, wherein first symbols indicate a geo-registered location of objects of interest and mission specific information.

9. The system of claim 1, wherein the first symbols indicate a location of friendly and non-friendly troops.

10. A method comprising:

capturing a first sensor image with a first sensor;

capturing a second sensor image with a second sensor;

merging the first sensor image with first symbols to provide a first image;

merging the second sensor image with second symbols to provide a second image;

providing the first image on a first waveguide combiner disposed in front of a first eye, the first image being provided conformally with a real world scene, wherein the first sensor is disposed above the first waveguide combiner; and

providing the second image on a second waveguide combiner disposed in front of a second eye, the second image being provided conformally with the real world scene, wherein the first symbols and second symbols provide geo-registered location of objects of interest and mission specific information, wherein the second sensor is disposed above the second waveguide combiner.

11. The method of claim 10, further comprising:

receiving wireless data related to a position of friendly and unfriendly troops.

12. The method of claim 10, wherein the first symbols and second symbols provide a zone of damage and a position of friendly and unfriendly troops.

13. The method of claim 10, wherein the first waveguide combiner has an input diffraction grating at a first outside area and an output grating configured for location in front of an eye.

14. The method of claim 10, wherein the first sensor and the second sensor are low light sensors configured to be disposed above a right eye and a left eye, respectively.

15. The method of claim 10, further comprising:

capturing a third sensor image with an infrared sensor disposed between the first sensor and the second sensor.

16. The method of claim 10, wherein the first waveguide combiner is disposed in a frame, and the second waveguide combiner is disposed in the frame, wherein a first projector is configured to provide the first image to the first waveguide combiner at a first outside peripheral portion of the frame, the first image and the real world scene being viewable by a first eye on the first waveguide combiner, wherein a second projector is configured to provide the second image to the second waveguide combiner at a second outside peripheral portion of the frame opposite the first outside peripheral portion, the second image and the real world scene being viewable by the second eye on the second waveguide combiner, and wherein a sensor module disposed substantially above the frame and comprises the first sensor, the second sensor, a video processing circuit, a symbol generator, a first image source and a second image source, the video processing circuit being configured to merge first sensor information from the first sensor and the first symbols from the symbol generator for the first image provided by the first image source, wherein the first symbols and the first sensor information are provided conformally with the real world scene, wherein the first image source is configured to provide the first image to the first projector, the video processing circuit being configured to merge second sensor information from the second sensor and the second symbols from the symbol generator for the second image provided by the second image source, wherein the second symbols and the second sensor information are provided conformally with the real world scene, wherein the second image source is configured to provide the second image to the second projector.

17. A helmet mounted display system, comprising:

a helmet; and

a vision system comprising;

a frame comprising a pair of waveguide combiners and a pair of projectors associated with the waveguide combiners, outputs of the projectors being disposed on opposite sides of the frame; and

a sensor module disposed substantially above the frame comprising a first sensor, a second sensor, a video processing circuit, and a symbol generator wherein the first sensor is disposed directly above one waveguide combiner, and the second sensor is disposed directly above the other waveguide combiner, the video processing circuit being configured to merge first sensor information from the first sensor and first symbols from the symbol generator for a first image provided by a first projector of the pair of projectors, wherein the first symbols and the first sensor information are provided conformally with a real world scene, the video processing circuit being configured to merge second sensor information from the second sensor and second symbols from the symbol generator for a second image provided by a second projector of the pair of the projectors, wherein the second symbols and the second sensor information are provided conformally with the real world scene.

18. The system of claim 17, wherein the sensor module comprises a thermal sensor disposed between the first sensor and the second sensor.

19. The system of claim 17, wherein the vision system is configured so that a portion between the frame and a temple of a user is not obstructed by the vision system.

20. The system of claim 17, wherein the vision system is slideably attached to the helmet from an operational position in front of a user'"'"'s eyes to a stowed position clear of the user'"'"'s eyes.