Intelligent range image camera for object measurement

• US 5,621,807 A
• Filed: 10/13/1994
• Issued: 04/15/1997
• Est. Priority Date: 06/21/1993
• Status: Expired due to Fees

• Current Assignee: EADS Deutschland GmbH (Airbus SE)
• Original Assignee: Dornier GmbH (Government of France)
• Inventors: Hopfmuller, Harald, Katzenbeisser, Rolf, Schaefer, Christoph, Eibert, Max
• Primary Examiner(s): Mancuso, Joseph
• Assistant Examiner(s): Patel, Jayanti K.
• Application Number:
  US08/322,343
• Time in Patent Office: 915 Days
• Field of Search: 382/107, 382/106, 382/108, 342/64, 364/424.02, 395/125, 395/129
• US Class Current: 382/103
• CPC Class Codes: G01S 17/89 for mapping or imaging

Abstract

A range imaging sensor for classifying and determining the position and orientation of remote objects combines a high resolution three-dimensional range imaging camera with a conventionally configured data processor. Three-dimensional object geometry sensed by the camera is compared with a plurality of three-dimensional models stored in a memory of the data processor. The spacial relationship between iteratively selected models and the sensed object is evaluated for varying positions and orientations of the model, and an optimum object classification and object position/orientation are detected and transmitted as an output signal.

Claims

1. Apparatus for detecting position and attitude of remotely sensed objects, comprising:

a sensor for acquiring three-dimensional surface contour image data of a remote object;

a memory having stored therein three-dimensional surface contour model data for a geometric model of at least one object;

object type hypothesis means for iteratively selecting one of said at least one geometric model and for reading said three-dimensional surface contour model data for said selected geometric model from said memory; and

a data processor for comparing said surface contour model data for said selected geometric model with said image data of said remote object and for generating output signals representative of an object classification, position and attitude for said remote object;

wherein said data processor includes a coarse position/attitude unit and a fine position/attitude unit, said coarse position/attitude unit comprising;

means for determining a two-dimensional partial horizontal contour of said remote object, as a function of said three-dimensional surface contour image data acquired by said sensor;

means for determining a two-dimensional complete horizontal contour of said geometric model, as a function of said surface contour model data stored in said memory;

means for determining a likelihood of a match of said partial horizontal contour and said complete horizontal contour independent of horizontal position and vertical rotation of said remote object, by comparing distances of three points on said partial horizontal contour with distances of three corresponding points on said complete horizontal contour;

means for iteratively selecting horizontal positions and vertical rotations of said remote object for maximization of said likelihood of a match;

means for identifying a horizontal position and vertical rotation for which said likelihood of a match is large and yields a true match of said partial and said complete horizontal contours; and

means for selecting a different geometric model if no spacial position and rotation yields a true match of said partial and said complete horizontal contours.

2. Apparatus according to claim 1 wherein said fine position/attitude unit comprises means for determining six variables of position and attitude of a specified object in the sensed image by minimizing an average distance between image points and the selected model surface, based on said horizontal position and vertical rotation.

3. Apparatus according to claim 2 wherein said hypothesized position/attitude determination comprises values for three-dimensional position of said remote object and values for its rotational orientation about three orthogonal axes, and wherein said output signal comprises an object classification based on said selected geometric model.

4. Apparatus according to claim 1 further comprising data transforming means for converting said surface contour image data acquired by said sensor into three-dimensional vector values for processing by said data processor.

5. Apparatus according to claim 4 wherein said fine position/attitude unit comprises means for comparing said data of said selected geometric model oriented in said hypothesized position/attitude with said three-dimensional surface contour image data acquired by said sensor, and for detecting a predetermined relationship between said data of said selected geometric model and said image data acquired by said sensor;

and

generating said output signal based on said hypothesized position/attitude in response to detection of said predetermined relationship.

6. Apparatus according to claim 1 further comprising:

an optical fiber of known length and intermittently coupled into an optical path of sensor; and

means for causing said sensor to measure time required for signals to propagate through said opticalfiber for calibrating said sensor.

7. Method of detecting position and attitude of remote object comprising the steps of:

scanning said remote object to acquire three-dimensional surface contour image data thereof;

selecting a geometric model from among a plurality of models of objects, and reading three-dimensional surface contour model data for said selected geometric model from a memory wherein said data are stored;

comparing said surface contour model data for said selected geometric model with said surface contour image data acquired in said scanning step;

detecting a predetermined relationship between said data for said selected geometric model and said data acquired by said scanner; and

generating output signals in response to detection of said predetermined relationship, said output signals being representative of an object classification and position and attitude of said remote object;

wherein said comparing step comprises;

determining a horizontal position and vertical rotation of a horizontal contour of said selected model such that horizontal contour points of the acquired remote object fit into the horizontal contour of said selected model as a subset;

determining a combined displacement of all points on said selected geometric model from points on the acquired three-dimensional surface contour of said remote object, for respective position and attitude hypotheses of said geometric model; and

determining a position and attitude of said geometric model for which said combined displacement is minimum; and

wherein said step of determining a position and attitude further comprises;

comparing said data of said selected geometric model oriented in said hypothesized position/attitude with said three-dimensional surface contour image data acquired from said scanning step to detect a predetermined relationship therebetween; and

generating said output signal based on said hypothesized position/attitude in response to detection of said predetermined relationship.

8. Apparatus according to claim 7 wherein said fine position/attitude unit comprises:

means for comparing said data of said selected geometric model oriented in said hypothesized position/attitude with said three-dimensional surface contour image data acquired by said sensor, and for detecting a predetermined relationship between said data of said selected geometric model and said image data acquired by said sensor; and

means for generating said output signal based on said hypothesized position/attitude in response to detection of said predetermined relationship.

9. Apparatus according to claim 7 wherein said hypothesized position/attitude determination comprises values for two-dimensional horizontal position of said remote object and its rotational orientation about a vertical axis, and wherein said output signal comprises an object classification based on said selected geometric model, values for three-dimensional position of said remote object and values for its rotational orientation about three orthogonal axes.

10. Apparatus according to claim 9 wherein said hypothesized position/attitude determination comprises values for two-dimensional horizontal position of said remote object and its rotational orientation about a vertical axis, and wherein said output signal comprises an object classification based on said selected geometric model, values for three-dimensional position of said remote object and values for its rotational orientation about three orthogonal axes.

11. Apparatus according to claim 7 further comprising data transforming means for converting said surface contour image data acquired by said sensor into three-dimensional vector values for processing by said data processor.

12. A machine vision system comprising:

a sensor for acquiring three-dimensional surface contour image data of a remote object;

a memory having stored therein three-dimensional surface contour model data for a geometric model of at least one object;

object type hypothesis means for iteratively selecting one of said at least one geometric model and for reading said three-dimensional surface contour model data for said selected geometric model from said memory;

a data processor for comparing said surface contour model data for said selected geometric model with said image data of said remote object and for generating output signals representative of an object classification and position and attitude for said remote object;

an interface unit for generating a control signal in response to said output signals; and

a controlled system actuated in response to said control signal;

wherein said data processor includes a coarse position/attitude unit and a fine position/attitude unit, said coarse position/attitude unit comprising;

means for determining a two-dimensional partial horizontal contour of said remote object, as a function of said three-dimensional surface contour image data acquired by said sensor;

means for determining a two-dimensional complete horizontal contour of said geometric model, as a function of said surface contour model data stored in said memory;

means for determining a likelihood of a match of said partial horizontal contour and said complete horizontal contour as a function of horizontal position and vertical rotation of said remote object, by comparing distances of three points on said partial horizontal contour with distances of three corresponding points on said complete horizontal contour;

means for iteratively selecting horizontal positions and vertical rotations of said remote object for maximization of said likelihood of a match;

means for identifying a horizontal position and vertical rotation for which said likelihood of a match is large and yields a true match of said partial and complete horizontal contours; and

means for selecting a different geometric model if no spacial position and rotate on yields a match of said partial and complete horizontal contours.

13. Apparatus according to claim 12 wherein said fine position/attitude unit comprises means for determining six variables of position and attitude of a specified object in the sensed image by minimizing an average distance between image points and the selected model surface based on said horizontal position and vertical rotation.

14. Apparatus according to claim 13 wherein said hypothesized position/attitude determination comprises values for three-dimensional position of said remote object and values for its rotational orientation about three orthogonal axes, and wherein said output signal comprises an object classification based on said selected geometric model.

15. Apparatus for detecting position and attitude of remotely sensed objects, comprising:

a sensor for acquiring three-dimensional surface contour image data of a remote object;

a memory having stored therein three-dimensional surface contour data for a geometric model of at least one object;

object type hypothesis means for iteratively selecting one of said at least one geometric model and for reading said three-dimensional surface contour data for said selected geometric model from said memory; and

a data processor for comparing said surface contour data for said selected geometric model with said image data of said remote object and for generating output signals representative of an object classification, position and attitude for said remote object;

wherein;

said data processor comprises a coarse position/attitude unit and a fine position/attitude unit, said coarse position/attitude unit comprising;

means for determining an orientation of said selected geometric model for which a displacement of all points on said selected geometric model from points on sensed surface contour of said image is minimized;

means for comparing said minimized difference with a predetermined limit value;

means for outputting a hypothesized position/attitude determination which corresponds to said orientation for which said displacement of all points is minimized, if said minimized difference is less than said predetermined limit value; and

means for actuating said object type hypothesis means to select a different one of said geometric models if said minimized difference is greater than said predetermined limit.

16. Method of detecting position and attitude of remote object comprising the steps of:

scanning said remote object to acquire three-dimensional surface contour image data thereof;

selecting a geometric model from among a plurality of models of objects, and reading three-dimensional surface contour data for said selected geometric model from a memory wherein said data are stored;

comparing said surface contour data for said selected geometric model with said surface contour image data acquired in said scanning step;

detecting a predetermined relationship between said data for said selected geometric model and said data acquired by said scanner; and

generating output signals in response to detection of said predetermined relationship, said output signals being representative of an object classification and position and attitude of said remote object;

wherein said comparing step comprises

determining a combined displacement of all points on said selected geometric model from points on the acquired three-dimensional surface contour of said remote object, for at least one position and orientation of said geometric model;

determining a position and orientation of said geometric model for which said combined displacement is minimum;

comparing said minimum combined displacement with a predetermined limit value; and

generating a hypothesized position/attitude determination which corresponds to said position and orientation having a minimum combined displacement when said minimum is less than said predetermined limit value; and

wherein said comparing step further comprises

comparing said data of said selected geometric model oriented in said hypothesized position/attitude with said three-dimensional surface contour image data acquired from said scanning step to detect a predetermined relationship therebetween; and

generating said output signal based on said hypothesized position/attitude in response to detection of said predetermined relationship.

17. A machine vision system comprising:

a sensor for acquiring three-dimensional surface contour image data of a remote object;

a memory having stored therein three-dimensional surface contour data for a geometric model of at least one object;

object type hypothesis means for iteratively selecting one of said at least one geometric model and for reading said three-dimensional surface contour data for said selected geometric model from said memory;

a data processor for comparing said surface contour data for said selected geometric model with said image data of said remote object and for generating output signals representative of an object classification and position and attitude for said remote object;

an interface unit for generating a control signal in response to said output signals; and

a controlled system actuated in response to said control signals;

wherein;

said data processor comprises a coarse position/attitude unit and a fine position/attitude unit, said coarse position/attitude unit comprising;

means for determining an orientation of said selected geometric model for which a displacement of all points on said selected geometric model from points on sensed surface contour of said image is minimized;

means for comparing said minimized difference with a predetermined limit value;

means for outputting a hypothesized position/attitude determination which corresponds to said orientation for which said displacement of all points is minimized, if said minimized difference is less than said predetermined limit value; and

means for actuating said object type hypothesis means to select a different one of said geometric models if said minimized difference is greater than said predetermined limit.