Using images to create measurements of structures through the videogrammetric process

• US 8,897,539 B2
• Filed: 10/04/2012
• Issued: 11/25/2014
• Est. Priority Date: 10/04/2011
• Status: Active Grant

• Current Assignee: Pointivo, Inc.
• Original Assignee: MetalForming, Inc.
• Inventors: Stone, Geoffrey, Wilkins, William Gaither, Fathi, Habib, Brilakis, Ioannis
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US13/644,523
• Publication Number:
  US 20130083990A1
• Time in Patent Office: 782 Days
• Field of Search: None
• US Class Current: 382/145
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/543 from line drawings
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06V 10/457 by analysing connectivity, e.g. edge linking, connected component analysis or slices
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects

Abstract

According to various embodiments, a stream of image frames depicting a structure in a scene are obtained. The stream of image frames may comprise first image frames from a first imaging device and second image frames from a second imaging device. Using the first image frames and the second image frames, a wireframe of at least a portion of the structure is generated. From the wireframe, as-built dimensions may be identified, materials estimates may be determined, and/or data for a fabrication device may be generated, for example.

Claims

1. A non-transitory computer-readable medium embodying a program executable in a computing device, the program comprising:

code that obtains a stream of image frames depicting a structure in a scene, the stream of image frames comprising a plurality of first image frames corresponding to a first imaging device and a plurality of second image frames corresponding to a second imaging device;

code that identifies a plurality of edge pixels in a plurality of the image frames;

code that identifies a plurality of groups of the edge pixels;

code that identifies a plurality of preliminary lines based at least in part on the groups of the edge pixels;

code that shortens the preliminary lines to thereby create a plurality of line segments;

code that extends the line segments to thereby create a plurality of extended line segments;

code that combines a plurality of the line segments to thereby create a plurality of lines;

code that identifies a plurality of potential planes in a plurality of the image frames;

code that matches a plurality of the lines for the first image frames with a plurality of the lines for the second image frames based at least in part on the potential planes to thereby identify a plurality of matched lines;

code that identifies a camera set motion for the image frames; and

code that generates a wireframe for at least a portion of the structure based at least in part on the matched lines and the camera set motion.

2. The non-transitory computer-readable medium of claim 1, wherein the program further comprises code that identifies a plurality of as-built dimensions for the structure based at least in part on the wireframe.

3. The non-transitory computer-readable medium of claim 1, wherein the program further comprises code that generates a materials estimate for the structure based at least in part on the wireframe.

4. The non-transitory computer-readable medium of claim 1, wherein the program further comprises code that generates data for a fabrication device based at least in part on the wireframe.

5. A system, comprising:

at least one computing device; and

a structure dimensioning system executable in the at least one computing device, the structure dimensioning system comprising;

logic that obtains a stream of image frames depicting a structure in a scene, the stream of image frames comprising a plurality of first image frames corresponding to a first imaging device and a plurality of second image frames corresponding to a second imaging device;

logic that generates a plurality of two-dimensional (2D) lines for at least a portion of the structure;

logic that generates a plurality of three-dimensional (3D) lines using at least a subset of the 2D lines, the 3D lines representing a plurality of physical edges depicted in the stream of image frames;

logic that identifies an imaging set motion for the stream of image frames;

logic that adjusts the 3D lines based at least in part on the imaging set motion to thereby generate a plurality of adjusted 3D lines; and

logic that generates a wireframe of at least a portion of the structure using at least a subset of the adjusted 3D lines.

6. The system of claim 5, wherein the structure dimensioning system further comprises logic that identifies a plurality of as-built dimensions for the structure based at least in part on the wireframe.

7. The system of claim 5, wherein the structure dimensioning system further comprises logic that generates a materials estimate for the structure based at least in part on the wireframe.

8. The system of claim 5, wherein the structure dimensioning system further comprises logic that generates data for a fabrication device based at least in part on the wireframe.

9. The system of claim 5, wherein the structure dimensioning system further comprises:

logic that identifies a plurality of edge pixels in at least a portion of the stream of image frames;

logic that identifies a plurality of groups of the edge pixels;

logic that identifies a plurality of preliminary lines based at least in part on the groups of the edge pixels; and

logic that generates the 2D lines based at least in part on at least a subset of the preliminary lines.

10. The system of claim 9, wherein the structure dimensioning system further comprises:

logic that shortens the preliminary lines to thereby create a plurality of line segments;

logic that extends the line segments based at least in part on a criteria to thereby create a plurality of extended line segments; and

logic that combines at least a subset of the extended line segments to thereby create the 2D lines.

11. The system of claim 5, wherein the structure dimensioning system further comprises:

logic that identifies a plurality of potential planes in at least a subset of the image frames; and

logic that generates the wireframe of at least the portion of the structure based at least in part on the potential planes.

12. A method, comprising:

obtaining, in at least one computing device, a stream of image frames depicting a structure in a scene, the stream of image frames comprising a plurality of first image frames corresponding to a first imaging device and a plurality of second image frames corresponding to a second imaging device;

generating, in the at least one computing device, a plurality of two-dimensional (2D) lines for at least a portion of the structure;

generating, in the at least one computing device, a plurality of three-dimensional (3D) lines using at least a subset of the 2D lines, the 3D lines representing a plurality of physical edges depicted in the stream of image frames;

identifying, in the at least one computing device, an imaging set motion for the stream of image frames;

adjusting, in the at least one computing device, the 3D lines based at least in part on the imaging set motion to thereby generate a plurality of adjusted 3D lines; and

generating, in the at least one computing device, a wireframe of at least a portion of the structure using at least a subset of the adjusted 3D lines.

13. The method of claim 12, further comprising identifying a plurality of as-built dimensions for the structure based at least in part on the wireframe.

14. The method of claim 12, further comprising generating a materials estimate for the structure based at least in part on the wireframe.

15. The method of claim 12, further comprising generating data for a fabrication device based at least in part on the wireframe.

16. The method of claim 12, further comprising:

identifying a plurality of edge pixels in at least a portion of the stream of image frames;

identifying a plurality of groups of the edge pixels;

identifying a plurality of preliminary lines based at least in part on the groups of the edge pixels; and

generating the 2D lines based at least in part on at least a subset of the preliminary lines.

17. The method of claim 16, wherein the groups of edge pixels are grouped based at least in part on a plurality of gradient angles associated with the edge pixels.

18. The method of claim 16, further comprising:

shortening, in the at least one computing device, the preliminary lines to thereby create a plurality of line segments;

extending, in the at least one computing device, the line segments based at least in part on a criteria to thereby create a plurality of extended line segments; and

combining, in the at least one computing device, at least a subset of the extended line segments to thereby create the 2D lines.

19. The method of claim 18, wherein the criteria for extending the line segments comprises a gradient angle threshold.

20. The method of claim 12, wherein generating the wireframe of at least a portion of the structure further comprises:

identifying a plurality of potential planes for a plurality of the image frames; and

generating the wireframe of at least a portion of the structure based at least in part on the potential planes.