COORDINATING IMAGE SENSING WITH MOTION

• US 20160073048A1
• Filed: 09/09/2014
• Published: 03/10/2016
• Est. Priority Date: 09/09/2014
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Howe, Wayne R.
• Granted Patent: US 9,781,378 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 9/8042 involving data reduction

Abstract

A method may include generating one or more control signals configured to control a velocity of an image sensing system based on a rate of recording pixels frames in the image sensing system. In some embodiments, the method may alternatively or additionally include generating one or more control signals configured to control the rate of recording in the image sensing system based on the velocity of the image sensing system. In some embodiments, a portion of image pixel data for a pixel frame may be output for transmission to a remote receiver. In some embodiments, the output data may include a recently recorded image data pixel for those surface resolution cells for which there is no currently stored reference image data pixel of a previously recorded pixel frame that is the same or is different by less than a threshold amount as the recently recorded image data pixel.

Claims

1. A method comprising:

generating one or more control signals configured to control a velocity of an image sensing system based on a rate of recording pixel frames in the image sensing system.

2. The method of claim 1, further comprising generating one or more control signals configured to control the rate of recording pixel frames in the image sensing system based on the velocity of the image sensing system.

3. The method of claim 1, further comprising generating one or more control signals configured to control a distance of the image sensing system from a surface being imaged by the image sensing system.

4. The method of claim 1, where the image sensing system is moving in a direction of travel relative to a surface being imaged that is spaced from the image sensing system, the image sensing system including one or more sensor elements for recording a pixel frame including at least M adjacent rows each having at least one image data pixel, with each image data pixel corresponding to a surface resolution cell of the surface determined by an instantaneous field of view of the surface by the associated sensor element, with the M rows disposed along the direction of travel, where M is an integer greater than one, and each surface resolution cell has an along-track dimension in the direction of travel, and wherein generating one or more control signals configured to control the velocity includes generating one or more control signals configured to control the velocity such that each recording interval of time associated with recording of the pixel frames is within a threshold amount of time equal to m multiplied by a time it takes for the image sensing system to move the along-track dimension relative to the surface, where m is an integer less than M.

5. The method of claim 4, further comprising outputting to a transmitter for transmission as base image data to a receiver that is remote from the image sensing system, a recorded image data pixel for each surface resolution cell from at least one recorded pixel frame and less than a number of recorded pixel frames that is equal to the highest integer less than M/m.

6. The method of claim 5, wherein outputting as base image data the recorded image data pixel for each cell includes outputting as base image data the recorded image data pixel for the surface resolution cell only one time.

7. The method of claim 6, wherein for each surface resolution cell, outputting as base image data, a recorded image data pixel includes outputting to the transmitter for transmission as base image data to the receiver, the recorded image data pixels corresponding to the first m rows of recorded image data pixels in the direction of travel.

8. The method of claim 2, where the image sensing system is moving in a direction of travel relative to a surface being imaged that is spaced from the image sensing system, the image sensing system including one or more sensor elements for recording a pixel frame including at least M adjacent rows each having at least one image data pixel, with each image data pixel corresponding to a surface resolution cell of the surface determined by an instantaneous field of view of the surface by the associated sensor element, with the M rows disposed along the direction of travel, where M is an integer greater than one, and each surface resolution cell has an along-track dimension in the direction of travel, wherein generating one or more control signals associated with one or more of controlling the velocity and controlling the rate of recording pixel frames, which one or more control signals are configured such that each recording interval of time associated with recording of the pixel frames is within a threshold amount of time equal to m multiplied by a time it takes for the image sensing system to move the along-track dimension relative to the surface, where m is an integer less than M, the method further comprising:

outputting to a transmitter for transmission as base image data to a receiver that is remote from the image sensing system, a recorded image data pixel for each surface resolution cell from at least one recorded pixel frame and less than a number of recorded pixel frames that is equal to the highest integer less than M/m; and

comparing, for each surface resolution cell for which base image data has been output, a recently recorded image data pixel with a previously recorded image data pixel, and outputting the recently recorded image data pixel to the transmitter for transmission to the receiver as change image data when the recently recorded image data pixel is different than the previously recorded image data pixel by at least a threshold amount.

9. The method of claim 1, wherein the image sensing system is moving in a direction of travel relative to a surface of the earth being imaged that is spaced from the image sensing system, the image sensing system including one or more sensor elements for recording a pixel frame including at least M adjacent rows each having at least one image data pixel, with each image data pixel corresponding to a surface resolution cell of the surface determined by an instantaneous field of view of the surface by the associated sensor element, with the M rows disposed along the direction of travel, where M is an integer greater than one, and each surface resolution cell has an along-track dimension in the direction of travel, the method further comprising, prior to generating the one or more control signals, determining the velocity and an altitude of the image sensing system relative to the surface of the earth using a satellite navigation system.

10. The method of claim 9, further comprising determining an orientation of the image sensing system relative to a nadir, determining a surface resolution cell size on the surface of the earth based at least in part on the determined altitude and image sensing system orientation, and determining the time it takes for the image sensing system to move the along-track dimension relative to the surface based at least in part on the determined velocity and surface resolution cell size.

11. The method of claim 10, where each of the pixel frames are produced by p rows of n sensor elements of the image sensing system, where p is a positive integer less than or equal to M, and n is a positive integer less than or equal to a number of image data pixels in each of the M rows, wherein generating one or more control signals includes generating one or more control signals such that each recording interval of time associated with recording of the pixel frames is within a threshold amount of time equal to m multiplied by a time it takes for the image sensing system to move the along-track dimension relative to the surface, where m is an integer less than M, the method further comprising outputting to a transmitter for transmission as base image data to a receiver that is remote from the image sensing system, a recorded image data pixel for each surface resolution cell from at least one recorded pixel frame and less than a number of recorded pixel frames that is equal to the highest integer less than M/m, wherein outputting base image data includes outputting for each recorded pixel frame, only base image data for surface resolution cells not previously output to the transmitter for transmission to the receiver.

12. The method of claim 11, further comprising comparing, for each surface resolution cell for which base image data has been output to the transmitter for transmission to the receiver, a recently recorded image data pixel with a previously output base or change image data, and outputting to the transmitter for transmission to the receiver as change image data the recently recorded image data pixel only if the recently recorded image data pixel is different than the previously recorded image data pixel by at least a threshold amount.

13. The method of claim 2, where the image sensing system is moving in a direction of travel relative to a surface being imaged that is spaced from the image sensing system, the image sensing system including p rows of n sensor elements for producing a pixel frame including at least M adjacent rows each having at least one image data pixel, with each image data pixel corresponding to a surface resolution cell of the surface determined by an instantaneous field of view of the surface by the associated sensor element, with the M rows disposed along the direction of travel, where M is an integer greater than one, and each surface resolution cell has an along-track dimension in the direction of travel, where p is a positive integer less than or equal to M, and n is a positive integer less than or equal to a number of image data pixels in each of the M rows, wherein generating one or more control signals associated with one or more of controlling the velocity and controlling the rate of recording pixel frames is configured such that each recording interval of time associated with recording of the pixel frames is within a threshold amount of time equal to m multiplied by a time it takes for the image sensing system to move the along-track dimension relative to the surface, where m is an integer less than M, the method further comprising outputting to a transmitter for transmission as base image data to a receiver that is remote from the image sensing system, a recorded image data pixel for each surface resolution cell from at least one recorded pixel frame and less than a number of recorded pixel frames that is equal to the highest integer less than M/m, wherein outputting base image data includes outputting only base image data from a pixel frame for surface resolution cells for which base image data has not previously been output to the transmitter for transmission to the receiver.

14. A method comprising:

generating one or more control signals configured to control a rate of recording pixel frames in an image sensing system based on a velocity of the image sensing system.

15. The method of claim 14, where the image sensing system is moving in a direction of travel relative to a surface being imaged that is spaced from the image sensing system, the image sensing system including one or more sensor elements for recording a pixel frame including at least M adjacent rows each having at least one image data pixel, with each image data pixel corresponding to a surface resolution cell of the surface determined by an instantaneous field of view of the surface by the associated sensor element, with the M rows disposed along the direction of travel, where M is an integer greater than one, and each surface resolution cell has an along-track dimension in the direction of travel, and wherein generating one or more control signals is configured such that each recording interval of time associated with recording of the pixel frames is within a threshold amount of time equal to m multiplied by a time it takes for the image sensing system to move the along-track dimension relative to the surface, where m is an integer less than M, the method further comprising outputting to a transmitter for transmission as base image data to a receiver that is remote from the image sensing system, a recorded image data pixel for each surface resolution cell from at least one recorded pixel frame and less than a number of recorded pixel frames that is equal to the highest integer less than M/m.

16. A method comprising:

storing a plurality of pixel frames taken from one or more sets of pixel frames received from an image sensing system moving in a direction of travel relative to a surface being imaged, wherein the surface is spaced from the image sensing system, the image sensing system having one or more sensor elements for producing the one or more sets of pixel frames, where each pixel frame includes at least M adjacent rows each having at least one image data pixel, each image data pixel corresponding to a surface resolution cell of the surface determined by an instantaneous field of view of the surface by the associated sensor element, the M rows being disposed along the direction of travel, M being an integer greater than one, and each surface resolution cell having an along-track dimension in the direction of travel, where each set of pixel frames includes pixel frames recorded at each of a plurality of sequential recording intervals of time, with each recording interval of time being within a threshold amount of time equal to m multiplied by a time it takes for the image sensing system to move the along-track dimension relative to the surface, where m is an integer less than M.

17. The method of claim 16, further comprising outputting to a transmitter for transmission to a receiver that is remote from the image sensing system, a recently recorded image data pixel for those surface resolution cells for which there is no currently stored reference image data pixel of a previously recorded pixel frame that is the same or is different by less than a threshold amount as the recently recorded image data pixel.

18. The method of claim 16, further comprising generating one or more control signals configured to control one or more of (a) a velocity of the image sensing system relative to the surface, (b) a distance of the image sensing system from the surface, and (c) a rate of recording the pixel frames so that each recording interval of time is within the threshold amount of time.

19. The method of claim 18, where the surface is the surface of the earth, the method further comprising, prior to generating one or more control signals, determining the velocity and altitude of the image sensing system relative to the surface of the earth using a satellite navigation system.

20. The method of claim 19, further comprising determining an orientation of the image sensing system relative to a nadir, determining a surface resolution cell size on the surface of the earth based at least in part on the determined altitude and image sensing system orientation, and determining the time it takes for the image sensing system to move the along-track dimension relative to the surface based at least in part on the determined velocity and surface resolution cell size.

21. A controller comprising:

a control component configured to be operatively coupled to an image sensing system, the image sensing system being supported by a platform disposed in spaced relationship from and moving in a direction of travel relative to a surface, the image sensing system having one or more sensor elements for producing one or more sets of pixel frames, where each pixel frame includes at least M adjacent rows each having at least one image data pixel, each image data pixel corresponding to a surface resolution cell of the surface determined by an instantaneous field of view of the surface by the associated sensor element, the M rows being disposed along the direction of travel, M being an integer greater than one, and each surface resolution cell having an along-track dimension in the direction of travel, the control component being configured to generate one or more control signals for controlling one or more of (a) a velocity of the image sensing system relative to the surface, (b) a distance of the image sensing system from the surface, and (c) a rate of recording the pixel frames, such that each recording interval of time associated with recording of the pixel frames is within a threshold amount of time equal to m multiplied by a time it takes for the image sensing system to move the along-track dimension relative to the surface, where m is an integer less than M.

22. The controller of claim 21, further comprising an output component coupled to the control component and configured to output to a transmitter for transmission as base image data from the controller to a receiver that is remote from the controller, an image data pixel for each surface resolution cell from at least one pixel frame and less than a number of pixel frames that is equal to the highest integer less than M/m.

23. The controller of claim 22, wherein the output component is further configured to output to the transmitter for transmission as base image data from the controller to the receiver, the image data pixel for each surface resolution cell only one time.

24. The controller of claim 22, wherein the output component is further configured, for each recorded pixel frame, to output to the transmitter for transmission as base image data the image data pixels of each pixel frame corresponding to the first m rows of image data pixels in the direction of travel.

25. The controller of claim 22, wherein the output component is further configured to compare a recently recorded image data pixel with a previously recorded image data pixel for each surface resolution cell for which base image data has been transmitted, and to output to the transmitter for transmission as change image data to the receiver, the recently recorded image data pixel when the recently recorded image data pixel is different than the previously recorded image data pixel by at least a threshold amount.

26. The controller of claim 21, where the surface is the surface of the earth, wherein the control component is further configured to determine the velocity and altitude of the image sensing system relative to the surface of the earth using a satellite navigation system, and to generate the one or more control signals based at least in part on the determined velocity and altitude.

27. The controller of claim 26, wherein the control component is further configured to determine an orientation of the image sensing system relative to a nadir, determine a surface resolution cell size on the surface of the earth based at least in part on the determined altitude and image sensing system orientation, and determine the time it takes for the image sensing system to move the along-track dimension relative to the surface based at least in part on the determined velocity and surface resolution cell size.

28. The controller of claim 22, where the image sensing system has p rows of n sensor elements that produce the pixel frames, where p is a positive integer less than or equal to M, and n is a positive integer less than or equal to a number of image data pixels in each of the M rows, and wherein the output component is further configured to output to the transmitter for transmission to the receiver, base image data from a pixel frame only for surface resolution cells for which base image data has not previously been transmitted.