Lidar-Assisted Stero Imager

• US 20100204974A1
• Filed: 02/09/2009
• Published: 08/12/2010
• Est. Priority Date: 02/09/2009
• Status: Abandoned Application

• Current Assignee: Utah State University
• Original Assignee: Utah State University
• Inventors: Pack, Robert T., Israelsen, Paul
• Application Number:
  US12/368,057
• Publication Number:
  US 20100204974A1
• Time in Patent Office: Days
• US Class Current: 703/17
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 19/40 Correcting position, velocity or attitude

Abstract

A lidar and one or more electro-optical (EO) imaging device may asynchronously acquire lidar shots and EO images. Navigation data comprising positioning, orientation, acceleration, and/or velocity information may be acquired as the lidar and EO data is captured. The lidar shots, EO images, and/or navigation data may be time stamped. The navigation and timing data may be used to associate a particular lidar shot and/or EO image with navigation data. The EO images may be captured at a higher capture rate and at a higher spatial resolution than the lidar shots. The navigation data may be used to cross correlate a lidar shot to a selected plurality of overlapping EO images. Ranging model information may be determined from EO image sequences using a stereo imaging technique. The stereo imaging technique may be seeded using the lidar shot data.

Claims

1. A system for asynchronously capturing correlatable lidar and EO imagery data to generate a model of a subject matter, the system comprising:

a computer-readable storage media;

a lidar;

an electro-optical (EO) imaging device;

a system controller comprising a processor, the system controller communicatively coupled to the computer-readable storage media, the lidar, and the EO imaging device,

wherein the system controller is configured to cause the lidar to capture a plurality of lidar shots of the subject matter at a lidar capture rate and to cause the EO imaging device to capture a plurality of overlapping EO images of the subject matter at an EO image capture rate, and

wherein the system controller is configured to acquire navigation data as the lidar shots and the EO images are captured; and

a modeler communicatively coupled to the system controller, the modeler configured to generate a model of the subject matter based on the plurality of overlapping EO images using a stereo imaging technique, wherein the stereo imaging technique is seeded using the lidar shots.

2. The system of claim 1, wherein the modeler is configured to seed the stereo imaging technique by mapping a lidar shot into a selected plurality of overlapping EO images.

3. The system of claim 1, wherein the modeler is configured to map the lidar shot into the selected plurality of overlapping EO images using the navigation data of the lidar shot and the navigation data of the plurality of overlapping EO images.

4. The system of claim 1, wherein the EO image capture rate is greater than the lidar capture rate.

5. The system of claim 1, wherein the EO images have a higher spatial resolution than the lidar shots.

6. The system of claim 1, further comprising a time source communicatively coupled to the system controller, wherein the system controller is configured to time stamp each of the plurality of lidar shots with a time the lidar shot was captured, and wherein the system controller is configured to time stamp each of the EO images with a time the EO image was captured.

7. The system of claim 6, further comprising:

a positioning system receiver communicatively coupled to the system controller, wherein the system controller is configured to acquire the navigation data using the positioning system receiver.

8. The system of claim 7, wherein the positioning system receiver is a global positioning system (GPS) receiver.

9. The system of claim 7, wherein the system controller is communicatively coupled to a secondary positioning system receiver disposed at a predetermined location, wherein the system controller is configured to refine the navigation data using positioning information received from the secondary positioning system receiver.

10. The system of claim 9, wherein the secondary positioning system receiver is a GPS receiver, and wherein the location of the secondary positioning system receiver is fixed.

11. The system of claim 7, wherein the system controller is configured to time stamp the navigation data with a time the navigation data was acquired.

12. The system of claim 11, wherein the modeler is configured to associate a particular lidar shot with corresponding navigation data using the time stamp of the particular lidar shot and the time stamp of the navigation data.

13. The system of claim 7, further comprising:

an inertial measurement unit (IMU) coupled to the lidar and the EO imaging device, the IMU communicatively coupled to the system controller, wherein the system controller is configured to refine the navigation data using the IMU.

14. The system of claim 13, wherein the IMU is configured to determine an orientation of the lidar and the EO imaging device, and wherein the navigation data comprises the orientation of the lidar and the orientation of the EO imaging device.

15. A method for asynchronously capturing correlatable lidar data and EO imagery data to develop a model of a subject matter therefrom, the method comprising:

a lidar capturing a plurality of lidar shots of the subject matter;

an EO imaging device capturing a plurality of overlapping EO images of the subject matter, wherein the lidar and the EO imaging device are configured to capture lidar shots and EO images asynchronously;

acquiring navigation data as the lidar shots and EO images are captured;

associating the plurality of lidar shots and the plurality of overlapping EO images with respective navigation data;

generating a model of the subject matter based on the plurality of overlapping EO images using a stereo imaging technique, wherein the stereo imaging technique is seeded using one or more of the plurality of lidar shots.

16. The method of claim 15, wherein the stereo imaging technique is seeded by mapping the one or more lidar shots into a selected plurality of overlapping EO images.

17. The method of claim 16, wherein mapping the lidar shot into the selected plurality of overlapping EO images comprises using the navigation data of the lidar shot and the navigation data of the plurality of overlapping EO images to map the lidar shot to respective image patches within the selected plurality of overlapping EO images.

18. The method of claim 17, wherein seeding the stereo imaging technique comprises seeding an image matching process using the lidar shot mappings.

19. The method of claim 15, wherein the navigation data is acquired from a positioning system receiver.

20. The method of claim 18, wherein the positioning system receiver is a global positioning system (GPS) receiver.

21. The method of claim 19, further comprising:

acquiring navigation data from a secondary positioning system receiver; and

refining the navigation data using the navigation data from the secondary positioning system receiver.

22. The method of claim 20, wherein the location of the secondary positioning system receiver is fixed.

23. The method of claim 19, further comprising:

acquiring an orientation of the lidar as each of the lidar shots is captured; and

acquiring an orientation of the EO imaging device as each of the EO images is captured, wherein the navigation data comprises the orientation of the lidar and the EO imaging device.

24. The method of claim 23, wherein the orientation of the EO imaging device is acquired from an inertial measurement unit (IMU) coupled to the EO imaging device.

25. The method of 24, further comprising:

time stamping each of the lidar shots with a time the lidar shot was captured;

time stamping the navigation data with a time the navigation data was acquired; and

associating a lidar shot with navigation data using the time stamp of the lidar shot and the time stamp of the navigation data.

26. The method of claim 25, further comprising time stamping each of the EO images with a time the EO image was captured;

and

associating an EO image with navigation data using the time stamp of the EO image and the time stamp of the navigation data.

27. A computer-readable storage medium comprising instructions to cause a computing device to perform a method for asynchronously capturing correlatable lidar and EO imagery data to generate a model of a subject matter, the method comprising:

a lidar capturing a plurality of lidar shots of the subject matter, wherein each of the lidar shots comprises a time stamp indicating the time the lidar shot was captured;

an EO imaging device capturing a plurality of overlapping EO images of the subject matter, wherein each of the EO images comprises a time stamp indicating the time the EO image was captured, wherein the lidar shots and EO images are captured asynchronously;

acquiring time stamped navigation data as the lidar shots and EO images are captured, wherein the navigation data comprises an orientation of the lidar and an orientation of the EO imaging device;

mapping a particular lidar shot onto a selected plurality of overlapping EO images using the navigation data; and

seeding a stereo imaging technique using the mapping.

28. A system for asynchronously capturing correlatable lidar and EO imagery data to generate a model of a subject matter, the system comprising:

a computer-readable storage media;

a lidar;

two or more electro-optical (EO) imaging devices;

a system controller comprising a processor, the system controller communicatively coupled to the computer-readable storage media, the lidar, and the two or more electro-optical (EO) imaging devices,

wherein the system controller is configured to cause the lidar to capture a plurality of lidar shots of the subject matter at a lidar capture rate and to cause the two or more electro-optical (EO) imaging devices to capture a plurality of overlapping EO images of the subject matter at an EO image capture rate, and

wherein the system controller is configured to acquire navigation data as the lidar shots and the EO images are captured; and

a modeler communicatively coupled to the system controller, the modeler configured to generate a model of the subject matter based on the plurality of overlapping EO images using a stereo imaging technique, wherein the stereo imaging technique is seeded using the lidar shots.