Systems and methods for overhead imaging and video

• US 8,487,996 B2
• Filed: 08/03/2012
• Issued: 07/16/2013
• Est. Priority Date: 04/25/2011
• Status: Active Grant

• Current Assignee: Planet Labs PBC
• Original Assignee: Skybox Imaging, Inc. (Planet Labs PBC)
• Inventors: Mann, Julian M., Berkenstock, Daniel, Robinson, M. Dirk
• Primary Examiner(s): Dastouri, Mehrdad
• Assistant Examiner(s): HALLENBECK-HUBER, JEREMIAH CHARLES
• Application Number:
  US13/566,110
• Publication Number:
  US 20120300064A1
• Time in Patent Office: 347 Days
• Field of Search: 348/135, 348/143, 348/144, 396/7
• US Class Current: 348/144
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G01C 11/025 by scanning the object
• CPC Class Codes: G01J 2003/2826 Multispectral imaging, e.g. filter imaging
• CPC Class Codes: G01J 3/2823 Imaging spectrometer
• CPC Class Codes: G01J 3/36 Investigating two or more bands of a spectrum by separate detectors
• CPC Class Codes: G01J 3/462 Computing operations in or between colour spaces; Colour management systems
• CPC Class Codes: G01J 3/513 having fixed filter-detector pairs
• CPC Class Codes: H01L 31/02162 for filtering or shielding light, e.g. multicolour filters for photodetectors
• CPC Class Codes: H04N 25/71 Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors

Abstract

Examples of an imaging sensor include a two-dimensional staring sensor with spectral filter strips for multispectral overhead imaging. The sensor may also include a panchromatic sensor with block or strip filters. The sensor may be used to collect multispectral color image data at a sampling resolution from overhead imaging platforms such as airplanes or satellites. The sensor can be used to provide video images. If a panchromatic sensor is included, the sensor may be used to collect panchromatic image data. Examples of methods for processing the image date include using the panchromatic image data to perform multi-frame enhancement or panchromatic sharpening on spectral images to improve their quality and resolution.

Claims

1. A sensor for collecting image data for use in an overhead imaging system that moves relative to a region of interest below the overhead imaging system, the sensor comprising:

an imaging sensor having an imaging surface area that has an imaging width, the imaging sensor configured to capture image data from the region of interest below the overhead imaging system at a frame capture rate, the region of interest having a plurality of features;

a plurality of spectral filters, the spectral filters in aggregate covering a spectral imaging portion of the imaging surface area of the imaging sensor,

each spectral filter having a spectral filter width at least substantially the same as the imaging width, and

each spectral filter having a spectral filter height greater than a minimum filter height,

wherein the minimum filter height equals a ratio of (1) a rate at which the region of interest appears to move across the imaging sensor due to relative motion of the overhead imaging system with respect to the region of interest to (2) the frame capture rate of the imaging sensor; and

a controller configured to control the imaging sensor to capture the image data at the frame capture rate, wherein a frame of the image data comprises a plurality of sub-images, each sub-image corresponding image data captured through one of the plurality of spectral filters, each sub-image including one or more of the plurality of features of the region of interest,

the controller further configured such that a portion of the region of interest is captured in a first sub-image of a first frame and said portion of the region of interest is captured in a second sub-image of a second frame, said second frame successive to said first frame, and said second sub-image corresponding to the same spectral filter as said first sub-image,

wherein the first frame and the second, successive frame are configured to be combined to form an output image by matching features from the portion of the region of interest present in the image data captured from the first sub-image and the second sub-image.

2. The sensor of claim 1, wherein a first spectral filter from the plurality of spectral filters has a greater spectral filter height than a second spectral filter from the plurality of spectral filters.

3. The sensor of claim 2, wherein the first spectral filter is a green spectral filter configured to transmit light within a range of wavelengths centered around green wavelengths or the first spectral filter is a blue spectral filter configured to transmit light within a range of wavelengths centered around blue wavelengths.

4. The sensor of claim 2, wherein the spectral filter height of the first spectral filter is at least twice the spectral filter height of the second spectral filter.

5. The sensor of claim 1, wherein the spectral filter heights of each of the plurality of spectral filters are between one and four times the minimum filter height.

6. The sensor of claim 1, wherein the plurality of spectral filters comprises at least two of a red spectral filter, a green spectral filter, a blue spectral filter, and an infrared spectral filter.

7. The sensor of claim 1, further comprising a panchromatic filter covering a panchromatic imaging portion of the imaging surface area of the imaging sensor, the panchromatic filter having:

a panchromatic filter width at least substantially the same as the imaging width,

a panchromatic filter height greater than or equal to the minimum filter height, and

a panchromatic bandwidth that is greater than spectral bandwidths of each of the spectral filters.

8. The sensor of claim 7, wherein the panchromatic filter height is greater than each of the spectral filter heights.

9. The sensor of claim 7, wherein the panchromatic filter height is substantially equal to a sum of all of the spectral filter heights.

10. The sensor of claim 7, wherein the panchromatic filter comprises two or more panchromatic filters, each panchromatic filter having a panchromatic filter height greater than or equal to the minimum filter height.

11. The sensor of claim 7, wherein the panchromatic bandwidth extends between about 450 nm and about 900 nm.

12. The sensor of claim 7, wherein the plurality of spectral filters comprises a blue spectral filter, a green spectral filter, a red spectral filter, and an infrared spectral filter.

13. The sensor of claim 7, a wherein the controller is configured to capture image data from the panchromatic imaging portion at a video frame rate and to provide the image data to a video feed.

14. The sensor of claim 13, wherein the video frame rate is greater than about 5 frames per second.

15. The sensor of claim 7, wherein a spatial resolution of the panchromatic imaging portion of the imaging surface area of the imaging sensor is substantially the same as a spatial resolution of the spectral imaging portion of the imaging surface area of the imaging sensor.

16. An imaging sensor configured to collect multispectral images from an overhead imaging platform that moves at an overhead speed relative to regions of interest on a surface below the imaging sensor, the imaging sensor comprising:

a light sensor comprising a plurality of light-sensitive pixels, the light sensor configured to capture images from the region of interest at an image capture rate, the region of interest having a plurality of features;

a plurality of spectral filters disposed relative to the light sensor such that light is received from the region of interest through the spectral filters before being collected by the light sensor, each spectral filter having a width and a height and capable of transmitting light in a spectral bandwidth,

wherein the height of each of the plurality of spectral filters is greater than a minimum height,

wherein the minimum height is equal to a ratio of the overhead speed to the image capture rate; and

a controller configured to control the light sensor to capture images at the image capture rate, wherein each image comprises a plurality of sub-images, each sub-image corresponding to light collected by the light sensor after being transmitted through one of the plurality of spectral filters each sub-image including light collected from one or more of the plurality of features of the region of interest,

the controller further configured such that light collected from a portion of the region of interest is captured in a first sub-image of a first image and said portion of the region of interest is captured in a second sub-image of a second image, said second image successive to said first image, and said second sub-image corresponding to light transmitted through the same spectral filter as said first sub-image,

wherein the first image and the second, successive image are configured to be combined to form an output image by matching features from the portion of the region of interest present in both the first sub-image and the second sub-image.

17. The imaging sensor of claim 16, wherein the light sensor comprises an active pixel sensor.

18. The imaging sensor of claim 16, wherein the plurality of spectral filters comprise at least three spectral filters, and combined spectral bandwidths of the at least three spectral filters include substantially the entire visible portion of the electromagnetic spectrum.

19. The imaging sensor of claim 16, further comprising a panchromatic filter disposed relative to the light sensor such that light is received from the region of interest through the panchromatic filter before being collected by the light sensor, the panchromatic filter having a panchromatic width and a panchromatic height and capable of transmitting light in a panchromatic bandwidth,

wherein the panchromatic bandwidth is greater than twice the spectral bandwidth of at least one of the spectral filters, and

wherein the panchromatic height is greater than or equal to the minimum height.

20. The imaging sensor of claim 19, wherein the panchromatic height is substantially equal to a sum of the heights of the spectral filters.