System and method for improving lidar data fidelity using pixel-aligned lidar/electro-optic data

• US 7,417,717 B2
• Filed: 10/05/2005
• Issued: 08/26/2008
• Est. Priority Date: 10/05/2005
• Status: Active Grant

• Current Assignee: Utah State University
• Original Assignee: Utah State University
• Inventors: Pack, Robert Taylor, Budge, Scott Ezra
• Primary Examiner(s): Tarcza; Thomas H.
• Assistant Examiner(s): Ratcliffe; Luke D
• Application Number:
  US11/244,212
• Publication Number:
  US 20070247612A1
• Time in Patent Office: 1,056 Days
• Field of Search: 356 401- 41, 356 301- 315, 356 501- 515, 356 6- 22
• US Class Current: 356/4.01
• CPC Class Codes: G01C 7/04 involving a vehicle which moves along the profile to be traced
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/89 for mapping or imaging

Abstract

A lidar and digital camera system collect data and generate a three-dimensional image. A lidar generates a laser beam to form a lidar shot and to receive a reflected laser beam to provide range data. A digital camera includes an array of pixels to receive optical radiation and provide electro-optical data. An optical bench passes the laser beam, reflected laser beam, and optical radiation and is positioned to align each pixel to known positions within the lidar shot. Pixels are matched to form a lidar point-cloud which is used to generate an image.

Claims

1. An apparatus for collecting data to generate a three-dimensional image, comprising:

a lidar to generate a laser beam that forms a lidar shot and to receive a reflected laser beam to thereby provide range data;

a digital camera having an array of pixels to receive optical radiation to thereby provide electro-optical data;

an optical bench to pass the laser beam, reflected laser beam, and optical radiation and positioned to align each pixel to known positions within the lidar shot;

a controller, including a processor, in electrical communication with the lidar and the digital camera; and

a memory in electrical communication with the processor and having stored thereon computer readable instruction code, to perform a method of;

obtaining a first lidar point-cloud and a second lidar point-cloud, each point-cloud comprising lidar range data and corresponding electro-optical data aligned to the respective lidar range data, and

merging the first lidar point-cloud and the second lidar point-cloud using the electro-optical data.

2. The apparatus of claim 1, wherein a resolution of the electro-optical data corresponding to the first lidar point-cloud is higher than a resolution of the lidar range data of the first lidar point-cloud.

3. The apparatus of claim 1, wherein the first lidar point-cloud cloud corresponds to a first area, and the second lidar point-cloud corresponds to a second area, wherein the first area and the second area are approximately the same.

4. The apparatus of claim 1, wherein the first lidar point-cloud corresponds to a first area, and the second lidar point-cloud corresponds to a second area, wherein the first area and the second area overlap one another.

5. The apparatus of claim 1, wherein merging the first lidar point-cloud with the second lidar point-cloud includes generating and applying corrections for errors in position, orientation, and/or time that occurred between obtaining the first and second point-clouds.

6. The apparatus of claim 1, further comprising a control interface in electrical communication with the controller to enable user interaction.

7. The apparatus of claim 1, further comprising a position and orientation system in electrical communication with the controller to generate position and orientation data.

8. The apparatus of claim 1, further comprising a communication interface in electrical communication with the controller to enable data communication.

15. The apparatus of claim 1, wherein the method further comprises image matching the electro-optical data of the first lidar point-cloud to the electro-optical data of the second point-cloud, and wherein the first lidar point-cloud is merged with the second lidar point-cloud using a result of the image matching.

9. An apparatus for collecting data to generate a three-dimensional image, comprising:

lidar means for generating a laser beam that forms a lidar shot and for receiving a reflected laser beam to thereby provide range data;

electro-optical means having an array of pixels for receiving optical radiation to thereby provide electro-optical data;

optical means for passing the laser beam, reflected laser beam, and optical radiation and for aligning each pixel to known positions within the lidar shot;

processor means in electrical communication with the lidar means and the electro-optical means; and

memory means in electrical communication with the processor means and having stored thereon computer readable instruction code, to perform a method of;

obtaining a first lidar point-cloud and a second lidar point-cloud, each point-cloud comprising lidar range data and corresponding electro-optical data, and

merging the first lidar point-cloud and the second lidar point-cloud using the electro-optical data.

10. The apparatus of claim 9, wherein a spatial resolution of the electro-optical data corresponding to the first lidar point-cloud is of a higher spatial resolution than the first lidar point-cloud, and wherein a spatial resolution of the electro-optical data corresponding to the second lidar point-cloud is of a higher spatial resolution than the second lidar-point-cloud.

11. The apparatus of claim 9, wherein the first lidar point-cloud approximately corresponds to the same area as the second lidar point-cloud.

12. The apparatus of claim 9, wherein the first lidar point-cloud corresponds to a first area and the second lidar point-cloud corresponds to a second area, wherein the first area and the second area overlap one another.

13. The apparatus of claim 9, wherein merging the first lidar point-cloud with the second lidar point-cloud includes generating and applying corrections for errors in position, orientation, and/or time that occurred between obtaining the first and second point-clouds.

14. The apparatus of claim 9, further comprising a position and orientation means in electrical communication with the processor means for generating position and orientation data.

16. The apparatus of claim 9, wherein the method further comprises image matching the electro-optical data of the first lidar point-cloud to the electro-optical data of the second point-cloud, and wherein the first lidar point-cloud is merged with the second lidar point-cloud using a result of the image matching.

17. A method for merging point-cloud data comprising corresponding pixel-aligned electro-optical data, comprising:

accessing a first lidar point-cloud and corresponding electro-optical data comprising a first plurality of lidar shots and a first plurality of pixels aligned to known positions relative to the first plurality of lidar shots;

accessing a second lidar point-cloud and corresponding electro-optical data comprising a second plurality of lidar shots and a second plurality of pixels aligned to known positions relative to the second plurality of lidar shots; and

matching the first plurality of pixels to the second plurality of pixels to thereby determine a mapping between the electro-optical data of the first lidar point-cloud and the electro-optical data of the second lidar point-cloud.

18. The method of claim 17, further comprising:

merging the first point-cloud and the second point-cloud using the mapping determined by matching the first plurality of pixels to the second plurality of pixels.

19. The method of claim 17, wherein the electro-optical data corresponding to the first lidar point-cloud is of a higher resolution than the first lidar point-cloud.

20. The method of claim 17, wherein the first point-cloud and the second point-cloud correspond to substantially the same scene.

21. The method of claim 17, wherein the first point-cloud and the second point-cloud overlap.

22. The method of claim 17, farther comprising:

merging a plurality of point-clouds using a mapping determined by image matching electro-optical data associated with each of the plurality of point-clouds with electro-optical data of overlapping point-clouds.

23. A computer-readable storage medium having stored thereon computer-readable instructions for performing a method of generating a three-dimensional image, the method comprising:

accessing a first lidar point-cloud and corresponding electro-optical data comprising a first plurality of lidar shots and a first plurality of pixels aligned to known positions relative to the first plurality of lidar shots;

accessing a second lidar point-cloud and corresponding electro-optical data comprising a second plurality of lidar shots and a second plurality of pixels aligned to known positions relative to the second plurality of lidar shots; and

matching the first plurality of pixels to the second plurality of pixels to thereby determine a mapping between the electro-optical data of the first lidar point-cloud and the electro-optical data of the second lidar point-cloud.

24. The computer-readable storage medium of claim 23, wherein the method farther comprises merging the first point-cloud and the second point-cloud using the mapping determined by matching the first plurality of pixels to the second plurality of pixels.

25. The computer-readable storage medium of claim 23, wherein the electro-optical data corresponding to the second lidar point-cloud is of a higher resolution than the second lidar point-cloud.

26. The computer-readable storage medium of claim 23, wherein the first point-cloud and the second point-cloud correspond to substantially the same scene.

27. The computer-readable storage medium of claim 23, wherein the first point-cloud and the second point-cloud overlap.

28. The computer-readable storage medium of claim 23, wherein the method farther comprises:

merging a plurality of point-clouds using a mapping determined by image matching electro-optical data associated with each of the plurality of point-clouds with electro-optical data of overlapping point-clouds.