Self-calibrated, remote imaging and data processing system

• US 8,483,960 B2
• Filed: 04/13/2010
• Issued: 07/09/2013
• Est. Priority Date: 09/20/2002
• Status: Active Grant

• Current Assignee: VI Technologies, LLC (VI GP LLC (f/k/a M7 Visual Intelligence GP LLC))
• Original Assignee: Visual Intelligence LP (f/k/a M7 Visual Intelligence LP) (VI GP LLC (f/k/a M7 Visual Intelligence GP LLC))
• Inventors: Smitherman, Chester L.
• Primary Examiner(s): Zanelli, Michael J
• Application Number:
  US12/798,899
• Publication Number:
  US 20100235095A1
• Time in Patent Office: 1,183 Days
• Field of Search: None
• US Class Current: 701/468
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/88 Lidar systems specially adapted for specific applications
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 19/14 specially adapted for specific applications
• CPC Class Codes: G01S 19/41 Differential correction, e.g. DGPS [differential GPS]
• CPC Class Codes: G01S 3/7803 Means for monitoring or calibrating
• CPC Class Codes: G01S 7/4811 common to transmitter and receiver
• CPC Class Codes: G01S 7/4972 Alignment of sensor
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils

Abstract

An imaging sensor system comprising: a rigid mount plate affixed to a vehicle; a first rigid mount unit affixed to the mount plate and having at least two imaging sensors disposed within the first mount unit, wherein a first imaging and a second imaging sensor each has a focal axis passing through an aperture in the first mount unit and the mount plate, wherein the first and second imaging sensor each generates a first array of pixels, wherein each array of pixels is at least two dimensional, wherein the first and second imaging sensors are offset to have a first image overlap area in the target area, wherein the first sensors image data bisects the second sensors image data in the first image overlap area.

Claims

1. A system for generating a map, comprising:

a global positioning receiver;

a vehicle in alignment with a target area;

an elevation measurement unit, in communication with the vehicle;

a global positioning antenna, in communication with the vehicle;

an attitude measurement unit, in communication with the vehicle;

an imaging sensor system, disposed to the vehicle, comprising;

a rigid mount plate affixed to the vehicle;

a first rigid mount unit affixed to the mount plate and having at least two imaging sensors disposed within the first mount unit, wherein a first imaging sensor and a second imaging sensor each has a focal axis passing through an aperture in the first mount unit and the mount plate, wherein the first and second imaging sensor each generates a first data array of pixels, wherein each data array of pixels is at least two dimensional, wherein the first and second imaging sensors are offset to have a first image overlap area in the target area, wherein the first sensors image data bisects the second sensors image data in the first image overlap area; and

a computer in communication with the elevation measurement unit, the global positioning antenna, the attitude measurement unit, the first imaging sensor, and the second imaging sensor;

correlating at least a portion of the image areas from the first imaging sensor and the second imaging sensor to a portion of the target area based on input from one or more of;

the elevation measurement unit, the global positioning antenna and the attitude measurement unit.

2. The system of claim 1 further comprising:

a third imaging sensor disposed within the first mount unit, wherein the third imaging sensor has a focal axis passing through the aperture in the first mount unit and the mount plate, wherein the third imaging sensor generates a third data array of pixels, wherein the third data array of pixels is at least two dimensional.

3. The system of claim 2, further comprising:

a fourth imaging sensor disposed within the first mount unit, wherein the fourth imaging sensor has a focal axis passing through an aperture in the first mount unit and the mount plate, wherein the fourth imaging sensor generates a fourth data array of pixels, wherein the fourth data array of pixels is at least two dimensional, wherein the third and fourth imaging sensors are offset to have a second image overlap area in the target area, wherein the third sensors image data bisects the fourth sensors image data in the second image overlap area.

4. The system of claim 3, wherein a first sensor array comprising the first and second image sensors and a second sensor array comprising the third and fourth image sensors are offset to have a third image overlap area in the target area, wherein the first sensor arrays image data bisects the second sensor arrays image data in the third overlap area.

5. The system of claim 3, wherein the first sensor arrays image data completely overlaps the second sensors arrays image data.

6. The system of claim 3, wherein third and fourth imaging sensors are selected from the group consisting of digital cameras, LIDAR, infrared, heat-sensing and gravitometers.

7. The system of claim 3, wherein the first and second imaging sensors are a digital camera and the third imaging sensor is a LIDAR.

8. The system of claim 2, wherein the third imaging sensor is selected from the group consisting of digital cameras, LIDAR, infrared, heat-sensing and gravitometers.

9. The system of claim 2, wherein the third imaging sensor is selected from the group consisting of a digital camera having a hyperspectral filter and a LIDAR.

10. The system of claim 2, wherein the first and second imaging sensors are a digital camera and the third imaging sensor is a LIDAR.

11. The system of claim 1, wherein the first mount unit and mount plate flex less than 100th of a degree during operation.

12. The system of claim 11, wherein the first mount unit and mount plate flex less than 1,000th of a degree during operation.

13. The system of claim 12, wherein the first mount unit and mount plate flex less than 10,000th of a degree during operation.

14. The system of claim 1, wherein the first imaging sensor is calibrated relative to one or more attitude measuring devices selected from the group consisting of a gyroscope, an IMU, and a GPS.

15. The system of claim 1, wherein the first and second imaging sensors are selected from the group consisting of digital cameras, LIDAR, infrared, heat-sensing and gravitometers.

16. An imaging sensor system comprising:

a rigid mount plate affixed to a vehicle in alignment with a target area;

a first rigid mount unit affixed to the mount plate and having at least two imaging sensors disposed within the first mount unit, wherein a first imaging sensor and a second imaging sensor each has a focal axis passing through an aperture in the first mount unit and the mount plate, wherein the first and second imaging sensor each generates a first data array of pixels, wherein each data array of pixels is at least two dimensional, wherein the first and second imaging sensors are offset to have a first image overlap area in the target area, wherein the first sensors image data bisects the second sensors image data in the first image overlap area.

17. The system of claim 16 further comprising:

a third imaging sensor disposed within the first mount unit, wherein the third imaging sensor has a focal axis passing through an aperture in the first mount unit and the mount plate, wherein the third imaging sensor generates a second data array of pixels, wherein the second data array of pixels is at least two dimensional.

18. The system of claim 17 further comprising:

a fourth imaging sensor disposed within the first mount unit, wherein the fourth imaging sensor has a focal axis passing through an aperture in the first mount unit and the mount plate, wherein the fourth imaging sensor generates a third data array of pixels, wherein the third data array of pixels is at least two dimensional, wherein the third and fourth imaging sensors are aligned and offset to have a second image overlap area in the target area, wherein the third sensors image data bisects the fourth sensors image in the second image overlap area.

19. The system of claim 18, wherein a first sensor array comprising the first and the second image sensor and a second sensor array comprising the third and the fourth image sensor are offset to have a third image overlap area in the target area, wherein first sensor arrays image data bisects the second sensor arrays image data in the third image overlap area.

20. The system of claim 18, wherein the first sensor arrays image data completely overlaps the second sensor arrays image data.

21. The system of claim 18, wherein the third and fourth imaging sensors are selected from the group consisting of digital cameras, LIDAR, infrared, heat-sensing and gravitometers.

22. The system of claim 18, wherein the first and second imaging sensors are a digital camera and the third imaging sensor is a LIDAR.

23. The system of claim 17, wherein the third imaging sensor is selected from the group consisting of digital cameras, LIDAR, infrared, heat-sensing and gravitometers.

24. The system of claim 17, wherein the third imaging sensor is selected from the group consisting of a digital camera having a hyperspectral filter and a LIDAR.

25. The system of claim 17, wherein the first and second imaging sensors are a digital camera and the third imaging sensor is a LIDAR.

26. The system of claim 16, wherein the first mount unit and mount plate flex less than 100th of a degree during operation.

27. The system of claim 26, wherein the first mount unit and mount plate flex less than 1,000th of a degree during operation.

28. The system of claim 27, wherein the first mount unit and mount plate flex less than 10,000th of a degree during operation.

29. The system of claim 16, wherein the first imaging sensor is calibrated relative to one or more attitude measuring devices selected from the group consisting of a gyroscope, an IMU, and a GPS.

30. The system of claim 16, wherein the first and second imaging sensors are selected from the group consisting of digital cameras, LIDAR, infrared, heat-sensing and gravitometers.

31. A method of calibrating imaging sensors comprising the steps of:

performing an initial calibration of the imaging sensors comprising;

determining the position of an AMU;

determining the position of a first imaging sensor within a first rigid mount unit relative to the AMU;

determining the position of a second imaging sensor within the first rigid mount unit relative to the AMU;

calibrating the first imaging sensor against a target area and determining a boresight angle of the first imaging sensor; and

calculating the position of one or more subsequent imaging sensors within the first rigid mount unit relative to the first imaging sensor; and

calibrating the one or more subsequent imaging sensors using the boresight angle of the first imaging sensor; and

using oversampling techniques to update at least one initial calibration parameter of the first imaging sensor against a target area and the boresight angle of the first imaging sensor;

using oversampling techniques to update the position of one or more subsequent imaging sensors within the first rigid mount unit relative to the first imaging sensor; and

updating at least one calibration parameter of one or more subsequent imaging sensors within the first rigid mount using the updated boresight angle of the first imaging sensor.

32. The method of claim 31, wherein the initial calibration step further comprises the steps of:

calibrating the second imaging sensor using the updated boresight angle of the first imaging sensor;

calculating the position of one or more subsequent imaging sensors within the first rigid mount unit relative to the first imaging sensor; and

calibrating the one or more subsequent imaging sensors within the first rigid mount using the updated boresight angle of the first imaging sensor.

33. The method of claim 32, further comprising the steps of:

using oversampling techniques to update the position of the second imaging sensor within the first rigid mount unit relative to the first imaging sensor;

using oversampling techniques to update the position of one or more subsequent imaging sensors within the first rigid mount unit relative to the first imaging sensor; and

updating at least one calibration parameter of one or more subsequent imaging sensors within the first rigid mount using the updated boresight angle of the first imaging sensor.

34. The method of claim 31, further comprising the steps of:

using flight line oversampling techniques to update the calibration of the first imaging sensor against a target area and the boresight angle of the first imaging sensor;

using flight line oversampling techniques to update the position of one or more subsequent imaging sensors within the first rigid mount unit relative to the first imaging sensor; and

updating at least one calibration parameter of one or more subsequent imaging sensors using the updated boresight angle of the first imaging sensor.

35. The method of 34, further comprising the steps of:

using flight line oversampling techniques to update the position of the second imaging sensor within the first rigid mount unit relative to the first imaging sensor;

using flight line oversampling techniques to update the position of one or more subsequent imaging sensors within the first rigid mount unit relative to the first imaging sensor; and

updating at least one calibration parameter of one or more subsequent imaging sensors within the first rigid mount using the updated boresight angle of the first imaging sensor.