Vehicle-mountable imaging systems and methods

• US 8,564,663 B2
• Filed: 04/14/2009
• Issued: 10/22/2013
• Est. Priority Date: 04/14/2009
• Status: Expired due to Fees

• Current Assignee: BAE Systems Plc
• Original Assignee: BAE Systems Information and Electronic Systems Integration Incorporated (BAE Systems Plc), BAE Systems Plc
• Inventors: Plemons, Danny, Breakfield, David K., Moore, David, Morris, Gary, Pybus, Charlie
• Primary Examiner(s): Zhang, Shirley
• Assistant Examiner(s): Nguyen, Angela
• Application Number:
  US12/423,194
• Publication Number:
  US 20100295945A1
• Time in Patent Office: 1,652 Days
• Field of Search: 348/148
• US Class Current: 348/148
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/103 using camera systems provided with artificial illumination device, e.g. IR light source
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/108 using 'non-standard' camera systems, e.g. camera sensor used for additional purposes i.a. rain sensor, camera sensor split in multiple image areas
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/60 characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60R 2300/8073 for vehicle security, e.g. parked vehicle surveillance, burglar detection
• CPC Class Codes: H04N 23/13 with multiple sensors
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2624 for obtaining an image which is composed of whole input images, e.g. splitscreen
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Imaging systems, methods, sensor pods, and vehicles having imaging systems. An imaging system can be coupled to a vehicle and can comprise: a plurality of imaging sensors with multiple spectrums; and a processing unit configured to generate an image model in substantially real-time by: warping, fusing, and stitching images; and two or more displays coupled to the processing unit such that the two or more displays can simultaneously each show a different portion of the image model, and such that a user can manually calibrate via the processing unit at least one of the warping, fusing, or stitching. In some imaging systems, imaging sensors are coupled at different elevations relative to a ground vehicle. The present sensor pods can comprise a housing; two or more first imaging sensors having a first spectrum; and two or more second imaging sensors having a second spectrum that is different than the first spectrum.

Claims

1. An imaging system for a ground vehicle, the imaging system comprising:

a plurality of first imaging sensors having a first spectrum, each first sensor having a field-of-view (FOV) and configured to be coupled to a vehicle such that the FOV partially overlaps the FOV of at least one of the other first imaging sensors;

a plurality of second imaging sensors having a second spectrum that is different than the first spectrum, each second sensor having a FOV and configured to be coupled to the vehicle such that the FOV is substantially aligned with the FOV of a corresponding first imaging sensor;

a processing unit configured to receive images from the first and second imaging sensors and to generate an image model in substantially real-time by;

warping images from corresponding first and second imaging sensors;

fusing images from corresponding first and second imaging sensors; and

stitching images from imaging sensors with partially overlapping FOVs; and

where the processing unit is configured to be coupled to two or more displays such that the two or more displays can simultaneously each show a different portion of the image model, and such that a user can manually calibrate via user inputs to the processing unit at least one of the warping, fusing, or stitching by which the processing unit is configured to generate the image model; and

where at least one of the first spectrum and the second spectrum includes;

infrared (IR), visible near-infrared (VNIR), image intensified (I²), or color VNIR.

2. The imaging system of claim 1, where the processing unit is configured to generate an image model by fusing images with an alpha-blending algorithm.

3. The imaging system of claim 1, where the processing unit is configured to generate an image model by warping images with one or more predefined warping templates each corresponding to an imaging sensor.

4. The imaging system of claim 1, where the processing unit is configured to generate an image model by fusing images with one or more predefined stitching templates.

5. The imaging system of claim 1, where the first imaging sensors are long-wavelength infrared (LWIR) imaging sensors.

6. The imaging system of claim 1, where the image model comprises images covering at least a continuous 180 degrees around the vehicle.

7. The imaging system of claim 1, where at least one of the first imaging sensors and the second imaging sensors are analog, and where the processing unit is further configured to convert analog images to digital images prior to generating the image model.

8. The imaging system of claim 1, where the imaging system is configured such that, if the processing unit is coupled to a display, the display can simultaneously show two or more different portions of the image model.

9. The imaging system of claim 1, where the imaging system is further configured such that, if the processing unit is coupled to the display, the display can also show a reference indicator indicative of the orientation of the portion of an image model shown on the display relative to the orientation of the vehicle.

10. The imaging system of claim 1, where the processing unit comprises one or more graphics processors.

11. The imaging system of claim 1, where the processing unit is configured to be coupled to two or more user-input devices each corresponding to a different display.

12. The imaging system of claim 1, further comprising:

a wireless communication device coupled to the processing unit and configured to transmit at least a portion of the image model to a remote receiver.

13. The imaging system of claim 5, where the second imaging sensors are color daylight (CD) imaging sensors.

14. The imaging system of claim 6, where the image model comprises images covering a continuous 360 degrees around the vehicle.

15. The imaging system of claim 8, where the imaging system is configured such that, if the processing unit is coupled to a display, the display can simultaneously show a front portion of an image model including images of in front of the vehicle and a mirrored rear portion of an image model including images of behind the vehicle.

16. The imaging system of claim 11, where processing unit is configured to be coupled to at least one user-input device that is integral to a corresponding display.

17. The imaging system of claim 11, where the imaging system is configured such that, if the processing unit is coupled to a display and a corresponding user-input device, a user can adjust, via the user-input device, the intensity of images from the first imaging sensors relative to the intensity of fused images from the second imaging sensors in a portion of the image model shown on the display.

18. The imaging system of claim 12, where the wireless communication device is further configured to receive signals from a remote user-input device.

19. The imaging system of claim 12, where the processing unit is further configured to compress at least a portion of the image model prior to the wireless communication device transmitting the at least a portion of the image model to a remote receiver.

20. The imaging system of claim 15 where the imaging system is configured such that, if the processing unit is coupled to the display, the display can also simultaneously show two side portions of an image model, each side portion including images from a different side of the vehicle.

21. A vehicle comprising:

a ground vehicle; and

an imaging system coupled to the vehicle, the imaging system comprising;

a plurality of first imaging sensors having a first spectrum, each first sensor having a field-of-view (FOV) and coupled to the vehicle such that the FOV partially overlaps the FOV of at least one of the other first imaging sensors;

a plurality of second imaging sensors having a second spectrum that is different than the first spectrum, each second sensor having a FOV and coupled to the vehicle such that the FOV is substantially aligned with the FOV of a corresponding first imaging sensor;

a processing unit configured to receive images from the first and second imaging sensors and to generate an image model in substantially real-time by;

warping images from corresponding first and second imaging sensors;

fusing images from corresponding first and second imaging sensors; and

stitching images from imaging sensors with partially overlapping FOVs; and

two or more displays coupled to the processing unit;

where the imaging system is configured such that the two or more displays can simultaneously each show a different portion of the image model, and such that a user can manually calibrate via user inputs to the processing unit at least one of the warping, fusing, or stitching by which the processing unit is configured to generate the image model; and

where at least one of the first spectrum and the second spectrum includes;

infrared (IR), visible near-infrared (VNIR), image intensified (I²), or color VNIR.

22. The vehicle of claim 21, where the ground vehicle is selected from the group consisting of:

M1117 Guardian Armored Security Vehicles (ASVs), High Mobility Multipurpose Wheeled Vehicles (Humvee), Family of Medium Tactical Vehicles (FMTV), Light Medium Tactical Vehicles (LMTV), Medium Tactical Vehicles (MTV), Medium Tactical Vehicle Replacements (MTVR), Heavy Expanded Mobility Tactical Trucks (HEMTT), Heavy Equipment Transport Systems (HETS), Palletized Load System (PLS) vehicles, and Bradley Fighting Vehicles.