Imager with multiple sensor arrays

• US 8,766,808 B2
• Filed: 03/08/2011
• Issued: 07/01/2014
• Est. Priority Date: 03/09/2010
• Status: Active Grant

• Current Assignee: Teledyne FLIR LLC (Teledyne Technologies Incorporated)
• Original Assignee: FLIR Systems, Inc. (Teledyne Technologies Incorporated)
• Inventors: Hogasten, Nicholas
• Primary Examiner(s): La, Anh V
• Application Number:
  US13/043,123
• Publication Number:
  US 20110221599A1
• Time in Patent Office: 1,211 Days
• Field of Search: 340/632, 340/633, 340/634, 340/584, 340555-557, 340/561, 340/565, 340/567, 348/E17.002, 348/E5.024, 348/E5.31, 348/E5.091, 348/164, 348/187, 348/222.1, 348/272, 348/294, 348/302, 348/218.1, 250/330, 250/332, 250/338.1, 250/338.4, 359/489.19, 359/589
• US Class Current: 340/632
• CPC Class Codes: H01L 27/14609 Pixel-elements with integrated switching, control, storage or amplification elements scanning details of imagers (circuitry of solid-state image sensors H04N25/00); circuitry of imagers H04N25/70
• CPC Class Codes: H01L 27/14621 Colour filter arrangements
• CPC Class Codes: H01L 27/14623 Optical shielding
• CPC Class Codes: H01L 27/14627 Microlenses
• CPC Class Codes: H01L 27/14685 Process for coatings or optical elements
• CPC Class Codes: H04N 13/218 using spatial multiplexing
• CPC Class Codes: H04N 25/671 for non-uniformity detection or correction
• CPC Class Codes: H04N 25/674 based on the scene itself, e.g. defocusing
• CPC Class Codes: H04N 25/68 applied to defects
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

An imager array may be provided as part of an imaging system. The imager array may include a plurality of sensor arrays (e.g., also referred to as lenslets or optical elements). Each sensor array may include a plurality of sensors (e.g., pixels) associated with a lens. The sensor arrays may be oriented, for example, substantially in a plane facing the same direction and configured to detect images from the same scene (e.g., target area). Such images may be processed in accordance with various techniques to provide images of electromagnetic radiation. The sensor arrays may include filters or lens coatings to selectively detect desired ranges of electromagnetic radiation. Such arrangements of sensor arrays in an imager array may be used to advantageous effect in a variety of different applications.

Claims

1. An imaging system comprising:

a housing;

an imager array disposed in the housing and adapted to image a scene, wherein the imager array comprises;

a plurality of sensor arrays, wherein each sensor array comprises;

a lens adapted to receive electromagnetic radiation from the seen, and

a plurality of sensors adapted to capture an image of the scene based on the electromagnetic radiation received through the lens,

the imager is adapted to capture a first image of the scene using at least a first one of the sensor arrays, and

the imager array is adapted to capture a second image of the scene using at least a second one of the sensor arrays substantially simultaneously with the first image, wherein the second image has a higher resolution than the first image;

a database comprising a mapping from the first image to the second image; and

a processor adapted to;

map a third image provided by the imager array to the first image,

wherein the third image has a resolution substantially equal to the first image, and

replace the third image with the second image to provide a high resolution result image.

2. The imaging system of claim 1, wherein each sensor array further comprises a filter adapted to pass only a subset of the electromagnetic radiation to the sensors.

3. The imaging system of claim 2, wherein the filter is a coating on the lens of the sensor array.

4. The imaging system of claim 1, wherein a third one of the sensor arrays comprises a first filter adapted to limit the electromagnetic radiation received by the lens of the third sensor array to a first spectral range, and a fourth one of the sensor arrays comprises a second filter adapted to limit the electromagnetic radiation received by the lens of the fourth sensor array to a second spectral range.

5. The imaging system of claim 1, wherein the sensor arrays are substantially equal in size.

6. The imaging system of claim 1, wherein at least a third one of the sensor arrays is greater in size than a fourth one of the sensor arrays, wherein the third sensor array is adapted to capture an image of the scene with higher resolution and a narrower field of view than the fourth sensor array.

7. The imaging system of claim 1, wherein the imager array further comprises a plurality of partitions adapted to block electromagnetic radiation outside a desired field of view of the sensors.

8. The imaging system of claim 1, wherein the imager array is a vacuum package and the lenses provide windows for the vacuum package.

9. The imaging system of claim 1, wherein at least two of the sensor arrays exhibit parallax relative to each other and are adapted to provide stereo images of the scene.

10. The imaging system of claim 1, wherein each sensor array comprises an optical axis extending from the lens to the sensors, wherein the optical axes of at least two of the sensor arrays exhibit phase shifts relative to each other, wherein images provided by the at least two sensor arrays are phase shifted relative to each other.

11. The imaging system of claim 10, further comprising a processor adapted to perform super resolution processing of the phase shifted images to provide a result image of the scene with a resolution greater than the resolution of the at least two sensor arrays.

12. The imaging system of claim 1, wherein one of the sensors in a third one of the sensor arrays is defective, wherein the imaging system further comprises a processor adapted to replace image data associated with the defective sensor with image data associated with a corresponding one of the sensors in a fourth one of the sensor arrays.

13. The imaging system of claim 1, further comprising a processor adapted to combine the images to provide a result image of the scene with a signal to noise ratio lower than signal to noise ratios of the images captured by the sensors.

14. The imaging system of claim 13, wherein the processor is adapted to:

map the sensors to a grid comprising a plurality of pixels;

accumulate signals from at least two of the sensor arrays for each of the grid pixels; and

provide the result image from the grid pixels.

15. The imaging system of claim 1, wherein the imaging system is a thermal camera.

16. A method of imaging, the method comprising:

receiving electromagnetic radiation from a scene at an imager array disposed in a housing of an imaging system, wherein the imager array comprises a plurality of sensor arrays, wherein each sensor array comprises a lens adapted to receive the electromagnetic radiation from the scene and a plurality of sensors adapted to capture an image of the scene based on the electromagnetic radiation received through the lens;

capturing a plurality of images of the scene substantially simultaneously using the sensors of the sensor arrays, wherein the capturing comprises;

capturing a first image of the scene using at least a first one of the sensor arrays, and

capturing a second image of the scene using at least a second one of the sensor arrays substantially simultaneously with the first image, wherein the second image has a higher resolution than the first image;

mapping the first image to the second image;

mapping a third image provided by the imager array to the first image, wherein the third image has a resolution substantially equal to the first image; and

replacing the third image with the second image to provide a high resolution result image.

17. The method of claim 16, wherein each sensor array further comprises a filter adapted to pass only a subset of the electromagnetic radiation to the sensors.

18. The method of claim 17, wherein the filter is a coating on the lens of the sensor array.

19. The method of claim 16, further comprising:

filtering the electromagnetic radiation received by the lens of a third one of the sensor arrays to a first spectral range; and

filtering the electromagnetic radiation received by the lens of a fourth one of the sensor arrays to a second spectral range.

20. The method of claim 16, wherein the sensor arrays are substantially equal in size.

21. The method of claim 16, wherein at least a third one of the sensor arrays is greater in size than a fourth one of the sensor arrays, wherein the image captured by the third sensor array has a higher resolution and a narrower field of view than the image captured by the fourth sensor array.

22. The method of claim 16, further comprising blocking electromagnetic radiation outside a desired field of view of the sensors using a plurality of partitions.

23. The method of claim 16, wherein the imager array is a vacuum package and the lenses provide windows for the vacuum package.

24. The method of claim 16, wherein at least two of the sensor arrays exhibit parallax relative to each other and their captured images are stereo images of the scene.

25. The method of claim 16, wherein each sensor array comprises an optical axis extending from the lens to the sensors, wherein the optical axes of at least two of the sensor arrays exhibit phase shifts relative to each other, wherein images provided by the at least two sensor arrays are phase shifted relative to each other.

26. The method of claim 25, further comprising performing super resolution processing of the phase shifted images to provide a result image of the scene with a resolution greater than the resolution of the at least two sensor arrays.

27. The method of claim 16, wherein one of the sensors in a third one of the sensor arrays is defective, the method further comprising replacing image data associated with the defective sensor with image data associated with a corresponding one of the sensors in a fourth one of the sensor arrays.

28. The method of claim 16, further comprising combining the images to provide a result image of the scene with a signal to noise ratio lower than signal to noise ratios of the images captured by the sensors.

29. The method of claim 28, further comprising:

mapping the sensors to a grid comprising a plurality of pixels;

accumulating signals from at least two of the sensor arrays for each of the grid pixels; and

providing the result image from the grid pixels.

30. The method of claim 16, wherein the method is performed by a thermal camera.

31. A gas detection system comprising:

an imager array adapted to image a scene, wherein the imager array comprises a plurality of sensor arrays, and wherein each sensor array comprises;

a lens adapted to receive electromagnetic radiation from the scene, and

a plurality of sensors adapted to capture an image of the scene based on the electromagnetic radiation received through the lens;

wherein a first one of the sensor arrays is adapted to capture a first image of a first wavelength range of the electromagnetic radiation;

wherein a second one of the sensor arrays is adapted to capture a second image of a second wavelength range of the electromagnetic radiation; and

wherein the second wavelength range is a subset of the first wavelength range and substantially corresponds to an absorption band of a gas.

32. The gas detection system of claim 31, further comprising a processor adapted to compare the first and second images to determine a presence of the gas in the scene.

33. The gas detection system of claim 32, wherein the processor is further adapted to;

map pixels of the first and second images to a coordinate space;

compare pixels of the first and second images mapped to the same coordinates in the coordinate space; and

determine the presence of the gas in the scene based on values of the pixels.

34. The gas detection system of claim 32, wherein the processor is further adapted to provide a result image that indicates the presence of the gas in the scene.

35. The gas detection system of claim 31, wherein the second sensor array comprises a filter adapted to pass only the second wavelength range of the electromagnetic radiation to the sensor.

36. The gas detection system of claim 35, wherein the filter is a coating on the lens of the second sensor array.

37. The gas detection system of claim 31, wherein a third one of the sensor arrays is adapted to capture a third wavelength range of the electromagnetic radiation, wherein the third wavelength range is a subset of the first wavelength range and substantially corresponds to another absorption band of the gas.

38. The gas detection system of claim 31, wherein a third one of the sensor arrays is adapted to capture a third wavelength range of the electromagnetic radiation, wherein the third wavelength range is a subset of the first wavelength range and substantially corresponds to an absorption band of another gas.

39. The gas detection system of claim 31, wherein the first and second sensor arrays are adapted to capture the first and second images substantially simultaneously.

40. The gas detection system of claim 31, wherein the gas detection system is a thermal camera further comprising a housing, wherein the imager array is disposed in the housing.

41. A method of detecting gas, the method comprising:

receiving electromagnetic radiation from a scene at an imager array, wherein the imager array comprises a plurality of sensor arrays, and wherein each sensor array comprises a lens adapted to receive the electromagnetic radiation from the scene and a plurality of sensors adapted to capture an image of the scene based on the electromagnetic radiation received through the lens;

capturing a first image of a first wavelength range of the electromagnetic radiation using a first one of the sensor arrays; and

capturing a second image of a second wavelength range of the electromagnetic radiation using a second one of the sensor arrays, wherein the second wavelength range is a subset of the first wavelength range and substantially corresponds to an absorption band of a gas.

42. The method of claim 41, further comprising comparing the first and second images to determine a presence of the gas in the scene.

43. The method of claim 42, wherein the comparing comprises:

mapping pixels of the first and second images to a coordinate space;

comparing pixels of the first and second images mapped to the same coordinates in the coordinate space; and

determining the presence of the gas in the scene based on values of the pixels.

44. The method of claim 42, further comprising providing a result image that indicates the presence of the gas in the scene.

45. The method of claim 41, wherein the second sensor array comprises a filter adapted to pass only the second wavelength range of the electromagnetic radiation to the sensor.

46. The method of claim 45, wherein the filter is a coating on the lens of the second sensor array.

47. The method of claim 41, further comprising capturing a third image of a third wavelength range of the electromagnetic radiation using a third one of the sensor arrays, wherein the third wavelength range is a subset of the first wavelength range and substantially corresponds to another absorption band of the gas.

48. The method of claim 41, further comprising capturing a third image of a third wavelength range of the electromagnetic radiation using a third one of the sensor arrays, wherein the third wavelength range is a subset of the first wavelength range and substantially corresponds to an absorption band of another gas.

49. The method of claim 41, wherein the capturing the first and second images are performed substantially simultaneously.

50. The method of claim 41, wherein the imager array is disposed in a housing of a thermal camera.

51. An imager calibration system comprising:

an imager array adapted to image a scene, wherein the imager array comprises a plurality of sensor arrays, and wherein each sensor array comprises;

a lens adapted to receive electromagnetic radiation from the scene, and

a plurality of sensors adapted to capture an image of the scene based on the electromagnetic radiation received through the lens; and

a processor adapted to;

receive a plurality of pixel values associated with the images captured by the sensors of the sensor arrays,

map the sensors to a coordinate space, wherein at least one sensor of each sensor array is mapped to each coordinate of the coordinate space, and

calculate an offset correction term for each sensor based on the pixel values of all sensors mapped to the same coordinate.

52. The imager calibration system of claim 51, wherein the imager calibration system is adapted to calibrate the imager array using the offset correction terms without obscuring the scene from view by the imager array.

53. The imager calibration system of claim 51, wherein the processor is adapted to:

accumulate the pixel values of all sensors mapped to each coordinate to provide an accumulated value for each coordinate; and

use the accumulated values to determine a mean value for each coordinate to calculate the offset correction terms.

54. The imager calibration system of claim 51, wherein the processor is adapted to;

determine differences between the pixel value of each sensor and the pixel values of all other sensors mapped to the same coordinate; and

accumulate the differences to calculate the offset correction terms.

55. The imager calibration system of claim 54, wherein the processor is adapted to map each coordinate to at least one sensor of each sensor array by horizontal and vertical offset values to determine the differences.

56. The imager calibration system of claim 51, wherein the processor is adapted to interpolate at least some of the pixel values to calculate at least one of the offset correction terms.

57. The imager calibration system of claim 51, wherein the processor is adapted to interpolate at least some of the coordinates to map at least one of the sensors to the coordinate space.

58. The imager calibration system of claim 51, wherein the processor is adapted to map the sensors to the coordinate space by horizontal and vertical offset values to account for variations among the sensor arrays.

59. The imager calibration system of claim 51, wherein the processor is adapted to adjust the map of the sensors to the coordinate space to compensate for parallax associated with variations among the sensor arrays.

60. The imager calibration system of claim 51, wherein the processor is adapted to:

update the offset correction terms; and

damp changes to the offset correction terms.

61. The imager calibration system of claim 51, wherein the processor is adapted adjust the offset correction terms to compensate for different sensitivities of the sensors.

62. The imager calibration system of claim 51, wherein the imager calibration system is a thermal camera further comprising a housing, wherein the imager array is disposed in the housing.

63. A method of calibrating an imaging system, the method comprising:

receiving electromagnetic radiation from a scene at an imager array, wherein the imager array comprises a plurality of sensor arrays, and wherein each sensor array comprises a lens adapted to receive the electromagnetic radiation from the scene and a plurality of sensors adapted to capture an image of the scene based on the electromagnetic radiation received through the lens;

receiving a plurality of pixel values associated with the images captured by the sensors of the sensor arrays;

mapping the sensors to a coordinate space, wherein at least one sensor of each sensor array is mapped to each coordinate of the coordinate space; and

calculating an offset correction term for each sensor based on the pixel values of all sensors mapped to the same coordinate.

64. The method of claim 63, wherein the method is performed without obscuring the scene from view by the imager array.

65. The method of claim 63, further comprising:

accumulating the pixel values of all sensors mapped to each coordinate to provide an accumulated value for each coordinate; and

using the accumulated values to determine a mean value for each coordinate to calculate the offset correction terms.

66. The method of claim 63, further comprising:

determining differences between the pixel value of each sensor and the pixel values of all other sensors mapped to the same coordinate; and

accumulating the differences to calculate the offset correction terms.

67. The method of claim 66, further comprising mapping each coordinate to at least one sensor of each sensor array by horizontal and vertical offset values to determine the differences.

68. The method of claim 63, further comprising interpolating at least some of the pixel values to calculate at least one of the offset correction terms.

69. The method of claim 63, further comprising interpolating at least some of the coordinates to map at least one of the sensors to the coordinate space.

70. The method of claim 63, wherein the mapping comprises mapping the sensors to the coordinate space by horizontal and vertical offset values to account for variations among the sensor arrays.

71. The method of claim 63, further comprising adjusting the mapping to compensate for parallax associated with variations among the sensor arrays.

72. The method of claim 63, further comprising:

updating the offset correction terms; and

damping changes to the offset correction terms.

73. The method of claim 63, further comprising adjusting the offset correction terms to compensate for different sensitivities of the sensors.

74. The method of claim 63, wherein the imager array is disposed in a housing of a thermal camera.