Controlling a line of sight angle of an imaging platform

• US 10,432,866 B2
• Filed: 08/08/2016
• Issued: 10/01/2019
• Est. Priority Date: 08/07/2015
• Status: Active Grant

• Current Assignee: Planet Labs PBC
• Original Assignee: Planet Labs Inc. (Planet Labs PBC)
• Inventors: Robinson, M. Dirk, Dyer, Jonathan, Levine, Joshua, Hermalyn, Brendan, Votel, Ronny, Messana, Matt William
• Primary Examiner(s): Wang, Xi
• Application Number:
  US15/230,785
• Publication Number:
  US 20170041548A1
• Time in Patent Office: 1,149 Days
• Field of Search: 3591981, 3591991, 3591994, 3592122, 3592141, 348 95, 2502011
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G02B 27/64 Imaging systems using optical elements for stabilisation of the lateral and angular position of the image
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/685 performed by mechanical compensation
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position

Abstract

Systems and methods of capturing imagery are provided. In particular, an imaging platform can be configured to capture imagery using a dual-axis steering mirror and one or more image capture devices. The line of sight angle of the imaging platform can be controlled by controlling the motion of the steering mirror in accordance with a motion profile. In example embodiments, the motion profile can correspond to a sawtooth wave. The imaging platform can further include one or more position sensors used to determine a position and/or orientation of the imaging platform along a path on which the imaging platform travels. The motion of the steering mirror can then be controlled to rotate about a first axis and a second axis to compensate for line of sight errors based at least in part on the determined position and/or orientation.

Claims

1. A computer-implemented method of controlling an imaging platform, the method comprising:

determining, by one or more computing devices, a motion profile for a dual-axis steering mirror associated with an imaging platform;

determining, by the one or more computing devices, position information indicative of an orientation of the imaging platform at one or more points along a path on which the imaging platform travels;

determining, by the one of more computing devices, a plurality of integration time periods based at least in part on the motion profile;

capturing, by the one or more computing devices, a sequence of image frames of at least a portion of a region of interest during at least a subset of the plurality of integration time periods as the imaging platform travels along the path;

identifying, by the one or more computing devices, blur in at least one of the captured image frames; and

controlling, by the one or more computing devices, the motion of the steering mirror based at least in part on the motion profile, the position information, and the identified blur, wherein controlling the motion of the steering mirror comprises controlling the steering mirror to rotate about a first axis and a second axis.

2. The computer-implemented method of claim 1, further comprising:

registering, by the one or more computing devices, the sequence of image frames to provide an image of the region of interest.

3. The computer-implemented method of claim 1, further comprising:

identifying, by the one or more computing devices, one or more misalignments between neighboring image frames in the sequence of captured image frames; and

controlling, by the one or more computing devices, the motion of the steering mirror based at least in part on the identified misalignments.

4. The computer-implemented method of claim 1, wherein each integration period corresponds to a period of time wherein relative motion between the imaging platform and the region of interest is reduced below a threshold.

5. The computer-implemented method of claim 1, wherein the motion profile comprises a sawtooth wave according to which the steering mirror is to be moved.

6. The computer-implemented method of claim 5, wherein controlling the motion of the steering mirror based at least in part on the motion profile comprises controlling the motion of the steering mirror such that a line of sight angle of the imaging platform travels from a first line of sight angle to a second line of sight angle at a first angular rate.

7. The computer-implemented method of claim 6, wherein controlling the motion of the steering mirror based at least in part on the motion profile further comprises, once the line of sight angle of the imaging platform reaches the second line of sight angle, controlling the motion of the steering mirror such that the line of sight of the imaging platform travels from the second line of sight angle to the first line of sight angle at a second angular rate.

8. The computer-implemented method of claim 7, wherein the second angular rate is a higher angular rate than the first angular rate.

9. A computing system, comprising:

one or more processors; and

one or more memory devices, the one or more memory devices storing computer-readable instructions that when executed by the one or more processors cause the one or more processors to perform operations, the operations comprising;

determining a motion profile for a dual-axis steering mirror associated with an imaging platform;

determining position information indicative of an orientation of the imaging platform at one or more points along a path on which the imaging platform travels;

determining a plurality of integration time periods based at least in part on the motion profile;

capturing a sequence of image frames of at least a portion of a region of interest during at least a subset of the plurality of integration time periods as the imaging platform travels along the path;

identifying blur in at least one of the captured image frames; and

controlling the motion of the steering mirror based at least in part on the motion profile, the position information, and the identified blur, wherein controlling the motion of the steering mirror comprises controlling the steering mirror to rotate about a first axis and a second axis.

10. The computing system of claim 9, wherein neighboring image frames in the sequence of image frames comprise overlapping measurements of the region of interest, the operations further comprising:

registering the sequence of image frames to provide a mosaicked image of the region of interest based at least in part on the overlapping measurements.

11. The computing system of claim 10, the operations further comprising:

identifying one or more misalignments between neighboring image frames in the sequence of captured image frames; and

controlling the motion of the steering mirror based at least in part on the identified misalignments.

12. The computing system of claim 9, wherein each integration period corresponds to a period of time wherein relative motion between the imaging platform and the region of interest is reduced below a threshold.

13. One or more tangible, non-transitory computer-readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising:

determining a motion profile for a dual-axis steering mirror associated with an imaging platform;

determining position information indicative of an orientation of the imaging platform at one or more points along a path on which the imaging platform travels;

determining a plurality of integration time periods based at least in part on the motion profile;

capturing a sequence of image frames of at least a portion of a region of interest during at least a subset of the plurality of integration time periods as the imaging platform travels along the path;

identifying blur in at least one of the captured image frames; and

controlling the motion of the steering mirror based at least in part on the motion profile, the position information, and the identified blur, wherein controlling the motion of the steering mirror comprises controlling the steering mirror to rotate about a first axis and a second axis.

14. The one or more tangible, non-transitory computer-readable media of claim 13, wherein neighboring image frames in the sequence of image frames comprise overlapping measurements of the region of interest, the operations further comprising:

registering the sequence of image frames to provide a mosaicked image of the region of interest based at least in part on the overlapping measurements.

15. The one or more tangible, non-transitory computer-readable media of claim 14, the operations further comprising:

identifying one or more misalignments between neighboring image frames in the sequence of captured image frames; and

controlling the motion of the steering mirror based at least in part on the identified misalignments.

16. The one or more tangible, non-transitory computer-readable media of claim 13, wherein controlling the motion of the steering mirror based at least in part on the motion profile comprises controlling the motion of the steering mirror such that a line of sight angle of the imaging platform travels from a first line of sight angle to a second line of sight angle at a first angular rate.

17. The one or more tangible, non-transitory computer-readable media of claim 13, wherein controlling the motion of the steering mirror based at least in part on the motion profile further comprises, once the line of sight angle of the imaging platform reaches the second line of sight angle, controlling the motion of the steering mirror such that the line of sight of the imaging platform travels from the second line of sight angle to the first line of sight angle at a second angular rate.

18. A computer-implemented method of controlling an imaging platform, the method comprising:

determining, by one or more computing devices, a motion profile for a dual-axis steering mirror associated with an imaging platform;

controlling, by the one or more computing devices, the motion of the steering mirror based at least in part on the motion profile;

determining, by the one or more computing devices, position information indicative of an orientation of the imaging platform at one or more points along a path on which the imaging platform travels;

controlling, by the one or more computing devices, the motion of the steering mirror based at least in part on the position information, wherein controlling the motion of the steering mirror comprises controlling the steering mirror to rotate about a first axis and a second axis;

determining, by the one or more computing devices, a plurality of integration time periods based at least in part on the motion profile, wherein each integration period corresponds to a period of time wherein relative motion between the imaging platform and a region of interest is reduced below a threshold; and

capturing, by the one or more computing devices, a sequence of image frames of at least a portion of the region of interest during at least a subset of the plurality of integration time periods as the imaging platform travels along the path.

19. A computing system, comprising:

one or more processors; and

one or more memory devices, the one or more memory devices storing computer-readable instructions that when executed by the one or more processors cause the one or more processors to perform operations, the operations comprising;

determining a motion profile for a dual-axis steering mirror associated with an imaging platform;

controlling the motion of the steering mirror based at least in part on the motion profile;

determining position information indicative of an orientation of the imaging platform at one or more points along a path on which the imaging platform travels;

controlling the motion of the steering mirror based at least in part on the position information, wherein controlling the motion of the steering mirror comprises controlling the steering mirror to rotate about a first axis and a second axis;

determining a plurality of integration time periods based at least in part on the motion profile, wherein each integration period corresponds to a period of time wherein relative motion between the imaging platform and a region of interest is reduced below a threshold; and

capturing a sequence of image frames of at least a portion of the region of interest during at least a subset of the plurality of integration time periods as the imaging platform travels along the path.

20. One or more tangible, non-transitory computer-readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising:

determining a motion profile for a dual-axis steering mirror associated with an imaging platform;

controlling the motion of the steering mirror based at least in part on the motion profile;

determining position information indicative of an orientation of the imaging platform at one or more points along a path on which the imaging platform travels;

controlling the motion of the steering mirror based at least in part on the position information, wherein controlling the motion of the steering mirror comprises controlling the steering mirror to rotate about a first axis and a second axis;

determining a plurality of integration time periods based at least in part on the motion profile, wherein each integration period corresponds to a period of time wherein relative motion between the imaging platform and a region of interest is reduced below a threshold; and

capturing a sequence of image frames of at least a portion of the region of interest during at least a subset of the plurality of integration time periods as the imaging platform travels along the path.