Method for reconstructing 3D lines from 2D lines in an image

• US 9,183,635 B2
• Filed: 05/20/2013
• Issued: 11/10/2015
• Est. Priority Date: 05/20/2013
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Ramalingam, Srikumar, Brand, Matthew
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US13/897,675
• Publication Number:
  US 20140341463A1
• Time in Patent Office: 904 Days
• Field of Search: 382/154
• US Class Current: 1/1
• CPC Class Codes: G06T 7/536 from perspective effects, e.g. by using vanishing points
• CPC Class Codes: G06T 7/543 from line drawings

Abstract

A method for reconstructing—three-dimensional (3D) lines in a 3D world coordinate system from two-dimensional (2D) lines in a single image of scene detects and clusters the 2D lines using vanishing points. A constraint graph of vertices and edges is generated, wherein the vertices represent the 2D lines, and the edges represents constraints on the 2D lines, then identifying the 3D lines that satisfy the constraints and reconstructing the 3D lines using the identified constraints.

Claims

1. A method for reconstructing-three-dimensional (3D) lines in a 3D world coordinate system from two-dimensional (2D) lines in a 2D image of scene, comprising:

detecting and clustering the 2D lines using vanishing points;

generating a graph of vertices and edges, wherein each vertex on the graph represents a line in the 2D image, and wherein each edge connecting two vertexes represents a possibility of a constraint on intersection of two lines corresponding to the two vertices in the 3D world coordinate system;

testing jointly the constraints on intersections using orthogonality, collinearity and junction properties of the intersection as a soft penalty for identifying the constraints on intersections that are satisfied on the 3D lines; and

reconstructing the 3D lines in the 3D world coordinate system using the identified constraints on intersections, wherein steps of the method are performed by a processor.

2. The method of claim 1, wherein the identifying uses linear programming.

3. The method of claim 2, wherein the penalty terms in the linear program uses L, X, T, W and Y junctions.

4. The method of claim 2, wherein a minimal spanning tree algorithm is used on the solution of the linear program to identify the satisfied constraints.

5. The method of claim 1, further comprising:

reconstructing the scene using the 3D lines.

6. The method of claim 5, wherein a camera is mounted on a household appliance for reconstructing an indoor scene.

7. The method of claim 6, wherein the indoor scene reconstruction is used for optimal airflow control in a heating, ventilation and air-conditioning unit.

8. The method of claim 1, wherein the reconstruction is applied to multiple images.

9. The method of claim 8, wherein a camera is mounted on a car and the large scene reconstruction is used detecting obstacles.

10. The method of claim 8, wherein a camera is mounted on a car and the large scene reconstructed is matched with 3D city models for computing the location of a car.

11. The method of claim 1, wherein the 3D lines are used for generating stereoscopic content.

12. The method of claim 1, wherein the 3D lines are used to provide 3D context for object detection and recognition.