Systems and methods for extracting planar features, matching the planar features, and estimating motion from the planar features

• US 8,340,400 B2
• Filed: 05/06/2009
• Issued: 12/25/2012
• Est. Priority Date: 05/06/2009
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Lukas, Jan, Kotaba, Ondrej
• Primary Examiner(s): Wang, Claire X
• Assistant Examiner(s): ROWLAND, JHERE M
• Application Number:
  US12/436,224
• Publication Number:
  US 20100284572A1
• Time in Patent Office: 1,329 Days
• Field of Search: None
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

A method of controlling an actuator based on a set of three-dimensional (3D) data points is provided. The method includes obtaining a first set of 3D data points for a scene and a second set of 3D data points for a scene with a sensor. At least a first set of planar features is extracted from the first set of 3D data point. At least a second set of planar features is extracted from the second set of 3D data points. A motion is determined between the first set of 3D data points and the second set of 3D data points based on a rotation and a translation from the at least a first set to the at least a second set. At least one actuator is controlled based on the motion.

Claims

1. A method of controlling an actuator based on a set of three-dimensional (3D) data points, the method comprising:

obtaining a first set of 3D data points for a scene and a second set of 3D data points for a scene with a sensor;

extracting at least a first set of planar features from the first set of 3D data points by determining a respective normal vector for each of the first set of 3D data points, wherein the respective normal vector for each of the first set of 3D data points is an estimation of a respective vector normal to the surface at that data point;

extracting at least a second set of planar features from the second set of 3D data points by determining a respective normal vector for each of the second set of 3D data points, wherein the respective normal vector for each of the second set of 3D data points is an estimation of a respective vector normal to the surface at that data point;

determining a motion between the first set of 3D data points and the second set of 3D data points based on a rotation and a translation from the at least a first set to the at least a second set; and

controlling at least one actuator based on the motion.

2. The method of claim 1, wherein extracting at least a first set of planar features further comprises:

determining an orthogonal distance from each data point of the first set of 3D data points to a defined location in the coordinate system;

grouping data points from the first set of 3D data points having similar normal vectors and orthogonal distances to form a first plurality of cells;

determining the at least of first set of planar features based on the first plurality of cells; and

wherein extracting at least a second set of planar features further comprises;

determining an orthogonal distance from each data point of the second set of 3D data points to a defined location in the coordinate system;

grouping data points from the second set of 3D data points having similar normal vectors assigned and orthogonal distances to form a second plurality of cells;

determining the at least a second set of planar features based on the second plurality of cells.

3. The method of claim 2, wherein determining up to a first plurality of planar features further comprises:

determining for each cell of the first plurality of cells whether the cell is a local maxima based on the number of data points within the cell; and

combining adjacent cells from the first plurality of cells that are local maxima; and

wherein determining up to a second plurality of planar features further comprises;

determining for each cell of the second plurality of cells whether the cell is a local maxima based on the number of data points within the cell; and

combining adjacent cells from the second plurality of cells that are local maxima.

4. The method of claim 2, wherein determining up to a first plurality of planar features further comprises:

determining a normal vector to represent each cell of the first plurality of cells;

determining an orthogonal distance to represent each cell of the first plurality of cells;

determining which of the first set of 3D data points are associated with a cell of the first plurality of cells based on similarities between the normal vector representing the cell of the first plurality of cells, the orthogonal distance representing the cell of the first plurality of cells and the normal vector and orthogonal distance for the first set of 3D data points; and

wherein determining up to a second plurality of planar features further comprises;

determining a normal vector to represent each cell of the second plurality of cells;

determining an orthogonal distance to represent each cell of the second plurality of cells;

determining which of the first set of 3D data points are associated with a cell of the second plurality of cells based on similarities between the normal vector representing the cell of the second plurality of cells, the orthogonal distance representing the cell of the second plurality of cells and the normal vector and orthogonal distance for the second set of 3D data points.

5. The method of claim 2, wherein determining a normal vector for a data point further comprises:

determining a mask for the data point, wherein the mask comprises a plurality of data points surrounding the data point including the data point; and

determining a first and a second tangential vectors from a gradient estimated using the mask, wherein the first tangential vector is a vector tangential to a surface of the mask in a first direction and the second tangential vector is a vector tangential to a surface of the mask in another direction;

wherein the normal vector is a cross product of the first and second tangential vectors.

6. The method of claim 1, wherein the sensor is located on a vehicle, the method further comprising:

displaying a position and orientation of the vehicle on a display device based on the determined motion.

7. The method of claim 1, further comprising:

generating a plurality of hypotheses from the at least a first set and the at least a second set;

evaluating each of the plurality of hypotheses; and

wherein determining a motion determines a motion based on the hypothesis having the best score.

8. A program product comprising a processor-readable medium on which program instructions are embodied, wherein the program instructions are operable to:

obtain a first set of 3D data points for a scene and a second set of 3D data points for a scene with a sensor;

extract at least a first set of planar features from the first set of 3D data points by determining a respective normal vector for each of the first set of 3D data points, wherein the respective normal vector for each of the first set of 3D data points is an estimation of a respective vector normal to the surface at that data point;

extract at least a second set of planar features from the second set of 3D data points by determining a respective normal vector for each of the second set of 3D data points, wherein the respective normal vector for each of the second set of 3D data points is an estimation of a respective vector normal to the surface at that data point;

determine a motion between the first set of 3D data points and the second set of 3D data points based on a rotation and a translation from the at least a first set to the at least a second set; and

display a position and orientation of the vehicle on a display device based on the determined motion.

9. The program product of claim 8, wherein when extracting at least a first set of planar features the program instructions are further operable to:

determine an orthogonal distance from each data point of the first set of 3D data points to a defined location in the coordinate system;

group data points from the first set of 3D data points having similar normal vectors and orthogonal distances to form a first plurality of cells;

determine the at least a first set of planar features from the first plurality of cells; and

wherein when extracting at least a first set of planar features the program instructions are further operable to;

determine a motion between the first set of 3D data points and the second set of 3D data points based on the first plurality of planar features and the second plurality of planar features;

determine an orthogonal distance from each data point of the second set of 3D data points to a defined location in the coordinate system;

group data points from the second set of 3D data points having similar normal vectors and orthogonal distances to form a second plurality of cells;

determine the at least a second plurality of planar features from the second plurality of cells.

10. The program product of claim 9, wherein when determining up to a first plurality of planar features the program instructions are further operable to:

determine a normal vector to represent each cell of the first plurality of cells;

determine an orthogonal distance to represent each cell of the first plurality of cells;

determine which of the first set of 3D data points are associated with a cell of the first plurality of cells based on similarities between the normal vector representing the cell of the first plurality of cells, the orthogonal distance representing the cell of the first plurality of cells and the normal vector and orthogonal distance for the first set of 3D data points; and

wherein when determining up to a second plurality of planar features the program instructions are further operable to;

determine a normal vector to represent each cell of the second plurality of cells;

determine an orthogonal distance to represent each cell of the second plurality of cells;

determine which of the first set of 3D data points are associated with a cell of the second plurality of cells based on similarities between the normal vector representing the cell of the second plurality of cells, the orthogonal distance representing the cell of the second plurality of cells and the normal vector and orthogonal distance for the second set of 3D data points.

11. The program product of claim 9, wherein when determining a normal vector for a data point the program instructions are further operable to:

determine a mask for the data point, wherein the mask comprises a plurality of data points surrounding the data point including the data point; and

determine a first and a second tangential vectors from a gradient estimated using the mask, wherein the first tangential vector is a vector tangential to a surface of the mask in a first direction and the second tangential vector is a vector tangential to a surface of the mask in another direction;

wherein the normal vector is a cross product of the first and second tangential vectors.

12. The program product of claim 8, wherein the program instructions are further operable to:

control at least one actuator based on the determined motion.

13. The program product of claim 8, wherein the program instructions are further operable to:

generate a plurality of hypotheses from the at least a first set and the at least a second set;

evaluate each of the plurality of hypotheses; and

wherein determining a motion determines a motion based on the hypothesis having the best score.

14. A system comprising:

a sensor operable to obtain a first set of three-dimensional (3D) data points for a scene and a second set of 3D data points for a scene; and

a processing unit coupled to the sensor;

a storage medium communicatively coupled to the processor from which the processor reads at least a portion of the software for execution thereby, wherein the software is configured to cause the processor to;

extract at least a first set of planar features from the first set of 3D data points by determining a respective normal vector for each of the first set of 3D data points, wherein the respective normal vector for each of the first set of 3D data points is an estimation of a respective vector normal to the surface at that data point;

extract at least a second set of planar features from the second set of 3D data points by determining a respective normal vector for each of the second set of 3D data points, wherein the respective normal vector for each of the second set of 3D data points is an estimation of a respective vector normal to the surface at that data point;

determine a motion between the first set of 3D data points and the second set of 3D data points based on a rotation and a translation from the at least a first set to the at least a second set; and

display a position and orientation of the vehicle on a display device based on the determined motion.

15. The system of claim 14, wherein when extracting at least a first set of planar features the processing unit is further configured to:

determine an orthogonal distance from each data point of the first set of 3D data points to a defined location in the coordinate system;

group data points from the first set of 3D data points having similar normal vectors and orthogonal distances to form a first plurality of cells;

determine the at least a first set of planar features from the first plurality of cells; and

wherein when extracting at least a first set of planar features the processing unit is further configured to;

determine a motion between the first set of 3D data points and the second set of 3D data points based on the first plurality of planar features and the second plurality of planar features;

determine an orthogonal distance from each data point of the second set of 3D data points to a defined location in the coordinate system;

group data points from the second set of 3D data points having similar normal vectors and orthogonal distances to form a second plurality of cells;

determine the at least a second plurality of planar features from the second plurality of cells.

16. The system of claim 15, when determining up to a first plurality of planar features the processing unit is further configured to:

determine for each cell of the first plurality of cells whether the cell is a local maxima based on the number of data points within the cell; and

combine adjacent cells from the first plurality of cells that are local maxima; and

wherein when determining up to a second plurality of planar features the processing unit is further configured to;

determine for each cell of the second plurality of cells whether the cell is a local maxima based on the number of data points within the cell; and

combine adjacent cells from the second plurality of cells that are local maxima.

17. The system of claim 15, wherein when determining up to a first plurality of planar features the processing unit is further configured to:

determine a normal vector to represent each cell of the first plurality of cells;

determine an orthogonal distance to represent each cell of the first plurality of cells;

determine which of the first set of 3D data points are associated with a cell of the first plurality of cells based on similarities between the normal vector representing the cell of the first plurality of cells, the orthogonal distance representing the cell of the first plurality of cells and the normal vector and orthogonal distance for the first set of 3D data points; and

wherein when determining up to a second plurality of planar features the processing unit is further configured to;

determine a normal vector to represent each cell of second plurality of cells;

determine an orthogonal distance to represent each cell of the second plurality of cells;

determine which of the first set of 3D data points are associated with a cell of the second plurality of cells based on similarities between the normal vector representing the cell of the second plurality of cells, the orthogonal distance representing the cell of the second plurality of cells and the normal vector and orthogonal distance for the second set of 3D data points.

18. The system of claim 15, wherein when determining a normal vector for a data point the processing unit is further configured to:

determine a mask for the data point, wherein the mask comprises a plurality of data points surrounding the data point including the data point; and

determine a first and a second tangential vectors from a gradient estimated using the mask, wherein the first tangential vector is a vector tangential to a surface of the mask in a first direction and the second tangential vector is a vector tangential to a surface of the mask in another direction;

wherein the normal vector is a cross product of the first and second tangential vectors.

19. The system of claim 14, wherein the processing unit is further configured to:

control at least one actuator based on the determined motion.

20. The system of claim 14, wherein the processing unit is further configured to:

generate a plurality of hypotheses from the at least a first set and the at least a second set;

evaluate each of the plurality of hypotheses; and

wherein determining a motion determines a motion based on the hypothesis having the best score.