System for generating three-dimensional electronic models of objects

• US 7,643,669 B2
• Filed: 06/30/2003
• Issued: 01/05/2010
• Est. Priority Date: 07/10/2002
• Status: Active Grant

• Current Assignee: Harman Becker Automotive Systems GmbH (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman Becker Automotive Systems GmbH (Samsung Electronics Co. Ltd.)
• Inventors: Nagel, Philipp Harald, Strassenburg-Kleciak, Marek
• Primary Examiner(s): Chawan; Sheela C
• Application Number:
  US10/519,252
• Publication Number:
  US 20060110026A1
• Time in Patent Office: 2,381 Days
• Field of Search: 382/154, 382/195, 382/284, 382/100, 364/449, 364/455, 364/456, 364/457, 364/460, 340/990, 340/995, 340/17, 345/7, 345/11, 345/23, 345/27, 345/419, 395/119, 395/124, 395/125, 395/127, 395/129, 700/98, 700/97, 700/118, 701/213
• US Class Current: 382/154
• CPC Class Codes: G01C 15/00 Surveying instruments or accessories not provided for in groups G01C1/00 - G01C13/00
• CPC Class Codes: G01C 15/002 Active optical surveying means optical plumbing G01C15/105
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/88 Lidar systems specially adapted for specific applications
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 19/14 specially adapted for specific applications
• CPC Class Codes: G01S 19/41 Differential correction, e.g. DGPS [differential GPS]
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

An image generation system for developing three-dimensional electronic models of objects includes a scanner and a computing system. The scanner may be selectively positioned adjacent to an object in different geographic positions. The scanner may provide scanner position information and image data representative of the object to the computing system. The computing system may generate a three-dimensional electronic model of the object from the image data and the scanner position information. The three-dimensional electronic model may be stored and utilized in other applications, such as an electronic navigational map.

Claims

1. An image generation system for developing three-dimensional electronic models of objects, the image generation system comprising:

a scanner operable to scan an object and provide scanner position data that includes a geographic position, slope and orientation of the scanner and image data representative of the object for each of a plurality of different geographic locations of the scanner in the vicinity of the object; and

a computing system in communication with the scanner, where the computing system is operable to generate a three-dimensional electronic model of the object by fitting together the image data provided from each of the geographic locations based on the scanner position data.

2. The image generation system of claim 1, where the scanner includes a satellite positioning system and a slope orientation sensor operable to provide the scanner position data.

3. The image generation system of claim 2, where the slope orientation sensor is operable to provide a pitch, a roll and an orientation of the scanner.

4. The image generation system of claim 1, where the scanner includes a laser scanner operable to provide geometric point data representative of a geometric shape of the object.

5. The image generation system of claim 1, where the scanner includes a point scanner and a color scanner, the point scanner and the color scanner operable to synchronously provide image data representative of a geometric shape and a color of the object.

6. The image generation system of claim 1, where the scanner includes a point scanner operable to collect geometric point data representative of the geometric shape of the object, a color scanner operable to collect color point data representative of the color of the object and a positioning system operable to collect the scanner position data.

7. The image generation system of claim 6, where the computing system is operable to associate the color point data, the geometric point data, and the scanner position data to form a three-dimensional electronic image representative of only one scan of the object.

8. The image generation system of claim 7, where the computing system is operable to selectively combine a plurality of three-dimensional electronic images as a function of the scanner position data to generate a three-dimensional electronic model.

9. The image generation system of claim 1, where the scanner position data includes navigational coordinates, an elevation, a facing direction of the scanner and a pitch and a roll of the scanner.

10. An image generation system for developing three-dimensional electronic models of objects, the image generation system comprising:

a point scanner operable to generate a plurality of point clouds representative of an object as a function of a plurality of respective scans of the object from a plurality of respective geographic positions around the object;

a color scanner synchronously operating with the point scanner, where the color scanner is operable to generate color point data representative of the color of the object for each of the point clouds;

a positioning system operable to provide geographic position data of the point scanner and the color scanner for each of the geographic positions; and

a computing system operable to develop a three-dimensional electronic model as a function of the point clouds, the color point data and the geographic position data.

11. The image generation system of claim 10, where the color scanner is a line sensor operable to measure a line of color in a determined area of the object, while at the same time the point scanner is operable to measure distances from a plurality of points within the same determined area.

12. The image generation system of claim 10, where the each of the point clouds includes geometric point data generated by the point scanner and the computing system is operable to synchronously capture the color point data and geometric point data as image data.

13. The image generation system of claim 12, where the geometric point data includes a geometric point and the color point data includes a color point and the color point is associated with the geometric point in the point cloud.

14. The image generation system of claim 10, where the computing system includes a site computing system and a lab computing system, the site computing system is operable to .perform a preliminary registration to form a preliminary three-dimensional electronic model, and the lab computing system is operable to perform precise registration of the preliminary three-dimensional electronic model to form a final three-dimensional electronic model.

15. The image generation system of claim 10, where the computing system includes a scan combining module and a geometric modeling module, the scan combining module operable to convert each of the point clouds to a plurality of lines that are exportable to the geometric modeling module, the geometric modeling module operable to form the three-dimensional electronic model from the lines.

16. The image generation system of claim 10, where the computing system is operable to develop a plurality of three-dimensional electronic images, each of the three-dimensional electronic images are developed as a function of one of the scans, where the three-dimensional electronic images are selectively combined to form the three-dimensional electronic model.

17. The image generation system of claim 16, where the computing system is operable to fill in gaps in color in a surface of each of the three-dimensional electronic images by division of the surface into triangles and color blending between color point data within each of the triangles.

18. The image generation system of claim 10, where the three-dimensional electronic model includes a simple layer, a construction layer, a position layer and a library layer.

19. The image generation system of claim 10, where the position system includes a satellite positioning system, a sensor configured to sense a facing direction and a sensor configured to sense a pitch and a roll of the sensor.

20. An image generation system for developing three-dimensional electronic models of objects, the image generation system comprising:

means for scanning an object, where the means for scanning is operable to determine position data that includes a geographic position, slope and orientation of the means for scanning an object and image data representative of the object for each of a plurality of different geographic locations of the means for scanning in the vicinity of the object; and

a computing system in communication with the means for scanning, where the computing system is operable to associate the position data with corresponding image data and dynamically fit together the image data from each of the different geographic locations based on the position data to generate a three-dimensional electronic model of the object.

21. The image generation system of claim 20, where the means for scanning includes means for determining navigational coordinates and means for determining slope, orientation and height of the means for scanning.

22. The image generation system of claim 20, where the computing system includes means for geometrically assembling the three-dimensional electronic model as a function of the navigational coordinates, the slope, orientation and height.

23. The image generation system of claim 20, where the computing system is configured to join a plurality of geometric points included in the image data, to form three-dimensional electronic images.

24. The image generation system of claim 23, where the computing system includes is configured to manipulate the three-dimensional electronic images.

25. The image generation system of claim 23, where the computing system is configured to combine the three-dimensional electronic images to form the three-dimensional electronic model.

26. The image generation system of claim 20, where the computing system includes is configured to texturize surfaces of the three-dimensional electronic model.

27. An image generation system for developing three-dimensional electronic models of objects, the image generation system comprising:

a memory device;

instructions in the memory device to store a plurality of three-dimensional electronic images, where each of the three-dimensional electronic images includes image data captured during a scan of an object

instructions stored in the memory device to associate position data indicative of a geographic location and orientation of a scanner used to capture the image data with the image data captured from the perspective of that geographic location;

instructions in the memory device to join a plurality of geometric points included in the image data of each of the three-dimensional electronic images; and

instructions in the memory device to combine the three-dimensional electronic images to form a three-dimensional electronic model as a function of the position data.

28. The image generation system of claim 27, further comprising instructions in the memory device to texturize the three-dimensional electronic model.

29. The image generation system of claim 27, where the instructions in the memory device to join geometric points comprise instructions in the memory device to form surfaces within each of the three-dimensional electronic images.

30. The image generation system of claim 27, where the instructions in the memory device to join the geometric points comprise instructions in the memory device to partition each of the three-dimensional electronic images into sub-images.

31. The image generation system of claim 30, where the instructions in the memory device to join the geometric points comprise instructions in the memory device to develop a plurality of lines, where each of the lines is representative of one of the sub-images.

32. The image generation system of claim 31, where the instructions in the memory device to combine the three-dimensional electronic images comprise instructions in the memory device to position the lines with respect to each other.

33. The image generation system of claim 31, where the image data also includes color points and instructions in the memory device to develop a plurality of lines comprises instructions in the memory device to convert the geometric points and the color points to a line that replaces the geometric points and the color points.

34. The image generation system of claim 27, where the instructions in the memory device to combine the three-dimensional electronic images comprise instructions in the memory device to positionally manipulate the three-dimensional electronic images with respect to each other.

35. A method of developing a three-dimensional electronic model representative of an object, the method comprising:

performing a scan of an object in each of a plurality of geographic positions with a scanner;

collecting image data and corresponding geographic position data of the scanner during each scan with a computer;

the computer developing a three-dimensional electronic image representative of each scan from the image data collected during the scan; and

the computer combining a plurality of three-dimensional electronic images as a function of the geographic position data that is associated with each of the three-dimensional electronic images to form a three-dimensional electronic model representative of the object.

36. The method of claim 35, where performing a scan comprises determining geometric points representative of the geometric shape of the object.

37. The method of claim 35, where performing a scan comprises determining color point data in a line of color within an area while determining geometric point data within the same area.

38. The method of claim 35, where collecting image data and position data comprises synchronously collecting geometric point data and color point data representative of the object.

39. The method of claim 35, where collecting image data and position data comprises determining navigational coordinates, a facing direction, a pitch, a roll and a height of a scanner used to perform the scan.

40. The method of claim 35, where combining a plurality of three-dimensional electronic images comprises associating the image data with the position data.

41. The method of claim 35, where combining a plurality of three-dimensional electronic images comprises manipulating at least a portion of the three-dimensional electronic images with respect to each other.

42. The method of claim 35, further comprising selecting a source texture as a function of a texture of the object;

developing a transformation procedure to create a complex texture from the source texture; and

associating the transformation procedure with a surface of the three-dimensional electronic model.

43. The method of claim 35, where the object includes a symmetrical portion and developing a three-dimensional electronic image comprises mirroring the image data from a scanned portion of the object to the symmetrical portion of the object.

44. A method of developing a three-dimensional electronic model representative of an object, the method comprising:

scanning an object with a scanner from a plurality of positions to develop a respective plurality of three-dimensional electronic images, where each of the three-dimensional electronic images is represented with a plurality of geometric points in a point cloud and a plurality of color points;

portioning one of the three-dimensional electronic images into a plurality of sub-images;

converting the geometric points and the color points into a plurality of lines representative of the respective sub-images; and

combining the lines to develop a three-dimensional electronic model.

45. The method of claim 44, where converting the geometric points and the color points comprises storing the lines representative of the sub-images as an outline data file.

46. The method of claim 44, where scanning an object comprises scanning from a plurality of positions to generate a plurality of respective point clouds.

47. The method of claim 44, where combining the lines comprises manipulating the lines with respect to each other to precisely fit together.

48. The method of claim 44, where combining the lines comprises minimizing error in the distance between one of the lines and another of the lines.

49. The method of claim 44, further comprising compensating for height when the resting surface that the object rests upon is sloped.

50. The method of claim 44, further comprising filling gaps in color in the three-dimensional electronic model by rasterization.

51. The method of claim 44, further comprising dividing a surface of each of the three-dimensional electronic images into triangles and color blending within each of the triangles to fill gaps in color.

52. The method of claim 44, where converting the geometric points and the color points comprises replacing the geometric points and the color points with the lines that are representative of the geometric points and color points.

53. A method of developing a three-dimensional electronic model representative of an object, the method comprising:

capturing a plurality of scans of an object with a scanner, where each of the scans includes image data representative of a three-dimensional electronic image and position data indicative of the position from which each scan was captured with the scanner;

combining the scans to form a three-dimensional electronic model of the object as a function of the position data; and

texturizing the three-dimensional electronic model as a function of a source texture that is an image file identified with a unique identifier and stored in a source texture library.

54. The method of claim 53, where texturizing comprises associating a texture with a surface of the three-dimensional electronic model.

55. The method of claim 53, where texturizing comprises selecting a source texture, creating a transformation procedure to transform the source texture to form a complex texture, and associating the transformation procedure with a surface of the three-dimensional electronic model.

56. The method of claim 53, further comprising organizing the three-dimensional electronic model into a structure that includes a plurality of layers.

57. The method of claim 53, where texturizing comprises searching the source texture library that includes a plurality of source textures, transforming a source texture to form a complex texture for a surface of the three-dimensional electronic model and storing in the library a transformation procedure to form the complex texture.

58. The method of claim 53, further comprising storing the three-dimensional electronic model as a datafile.