Method to control a drive mechanism of an automated machine having a camera

• US 10,234,260 B2
• Filed: 04/29/2016
• Issued: 03/19/2019
• Est. Priority Date: 06/12/2015
• Status: Active Grant

• Current Assignee: Hexagon Technology Center GmbH (Hexagon AB)
• Original Assignee: Hexagon Technology Center GmbH (Hexagon AB)
• Inventors: Siercks, Knut, Jordil, Pascal
• Primary Examiner(s): Kalapodas, Dramos
• Application Number:
  US15/143,414
• Publication Number:
  US 20160364869A1
• Time in Patent Office: 1,054 Days
• CPC Class Codes: B25J 9/1666 Avoiding collision or forbidden zones
• CPC Class Codes: G01B 11/002 for measuring two or more coordinates
• CPC Class Codes: G01B 21/047 Accessories, e.g. for positioning, for tool-setting, for measuring probes
• CPC Class Codes: G01B 5/008 using coordinate measuring machines
• CPC Class Codes: G01B 7/012 Contact-making feeler heads therefor
• CPC Class Codes: G06K 7/1413 1D bar codes
• CPC Class Codes: G06K 7/1417 2D bar codes
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

The invention relates to a method to control a drive mechanism of an automated machine for approaching a point of interest on an object, the object having a surface on which a reference information of the object is provided as a one- or two-dimensional pattern, the method comprising capturing at least one image of the object by means of at least one camera of the automated machine, identifying the reference information in the at least one image, determining a position and orientation of the reference information relative to the at least one camera, determining a position and orientation of the object based on the determined position and orientation of the reference information, and controlling the drive mechanism based on the determined position and orientation of the object.

Claims

1. A method to control a drive mechanism of an automated machine for approaching a point of interest on an object, the object having a surface on which reference information of the object is provided as a one- or two-dimensional pattern, the method comprising:

capturing at least one image of the object using at least one camera of the automated machine, wherein each of the at least one cameras is arranged at a predetermined known position with the automated machine;

identifying the pattern in the at least one image;

determining before starting the measurement a position and orientation of the pattern relative to the at least one camera by image recognition, wherein identifying the pattern comprises reading position information related to a position and orientation of the pattern on the object, the position information being content of the reference information encoded in the pattern or is extracted from a data base of a data base storage device as part of an object data set comprising reference information of the object and being assignable to the object by the reference information encoded in the pattern;

determining a position and orientation of the object based on the position information related to the position and orientation of the pattern on the object and the determined position and orientation of the pattern relative to the at least one camera; and

controlling the drive mechanism based on the determined position and orientation of the object.

2. The method according to claim 1, wherein:

the automated machine is a coordinate measuring machine,

the point of interest is a measuring point on an object to be measured by the coordinate measuring machine,

the drive mechanism is adapted to drive a probe head of the coordinate measurement machine in a manner such that the probe head is capable to approach the measurement point, and

the coordinate measuring machine is adapted for determining at least one spatial coordinate of the measurement point.

3. The method according to claim 1,

wherein the image recognition is configured to identify the pattern provided on the object in one of the following forms:

engraved on the object;

laser-engraved on the object;

printed on the object;

or

on a sticker and applying the sticker on the object,

wherein the pattern comprising the reference information is provided as a barcode or QR code.

4. The method according to claim 1, wherein determining the position and orientation of the pattern relative to the at least one camera comprises determining a distortion of the pattern by means of image recognition.

5. The method according to claim 1, wherein:

the pattern as part of the reference information comprises object-related information, information about the object type, a serial number, or dimensions of the object,

identifying the pattern comprises reading the object-related information, and

the object-related information is encoded in the reference information and pattern, respectively, and reading the object-related information comprises decoding the pattern and its respective reference information, respectively.

6. The method according to claim 1, wherein:

providing object data comprising data sets of a plurality of objects from the data base storage device configured to store and provide object data, each data set comprising the pattern of an object and each data set comprising the reference information of an object, wherein the reference information comprises object-related information, information about the object type, a serial number, or dimensions of the object.

9. The method according to claim 1, wherein:

an image of the object is presented to a user, and

wherein the method comprises an augmented reality step, in the course of which digital information of the object is presented in the image of the object, wherein the position and orientation of the digital information in the image relative to the object is based on the determined position and orientation of the pattern.

10. The method according to claim 9, wherein the digital information comprises CAD data of the object and/or is superposed on the object in the image.

20. The method according to claim 1, wherein the pattern is a QR-code.

21. The method according to claim 1, wherein the pattern is a barcode.

7. A method to control a drive mechanism of an automated machine, wherein the automated machine is for approaching a point of interest on an object, the object having a surface on which a reference information of the object is provided as a one- or two-dimensional pattern, the method comprising:

capturing at least one image of the object using at least one camera of the automated machine, wherein each of the at least one cameras is arranged at a predetermined known position in/at/on the automated machine;

identifying the pattern in the at least one image;

determining before starting the measurement a position and orientation of the pattern relative to the at least one camera by image recognition; and

providing a data set of the object from a storage device of the machine adapted to store and provide object data, the object data comprising data sets of a plurality of objects, the provided data set comprising reference information of the object and being assignable to the object by the reference information encoded in the pattern, wherein

the pattern comprises position information related to a position and orientation of the pattern on the object, the position information being content of the reference information or is extracted from the data base of the storage device as part of the object data set,

determining a position and orientation of the object based on the position and orientation of the pattern relative to the at least one camera and on the position information related to the position and orientation of the pattern on the object; and

controlling the drive mechanism based on the determined position and orientation of the object.

8. The method according to claim 7, further comprising determining the position and orientation of the pattern relative to the at least one camera comprises determining a distortion of the pattern by means of image recognition.

22. The method according to claim 7, wherein:

an image of the object is presented to a user, and

wherein the method comprises an augmented reality step, in the course of which digital information of the object is presented in the image of the object, wherein the position and orientation of the digital information in the image relative to the object is based on the determined position and orientation of the pattern.

23. The method according to claim 7, wherein the reference information of an object included in a data set of an object comprising object-related information, information about the object type, a serial number, or dimensions of the object.

11. An automated machine comprising:

a base;

a drive mechanism for approaching a point of interest on an object;

a frame structure being movable in a horizontal and a vertical direction, wherein the base, the drive mechanism, and the frame structure define a measuring volume inside which the at least one spatial coordinate of the measurement point is determinable, and the automated machine further comprises;

at least one camera that is adapted to be directed to the measuring volume for providing at least a first image of at least a first part of the measuring volume, wherein each of the at least one cameras having a predetermined known position with the automated machine; and

a controller adapted to control the drive mechanism on the basis of image data derived from the at least first image;

wherein the at least one camera is adapted;

to identify in the at least first image a pattern on an object in the measuring volume, and

to determine before starting a measurement a position and orientation of the pattern relative to the at least one camera by image recognition; and

wherein the controller is adapted to determine a position and orientation of the object based on the position and orientation of the pattern relative to the camera and on a provided position information related to a position and orientation of the pattern on the object to control the drive mechanism on the basis of the determined position and orientation of the object,

wherein the identifying the pattern comprises reading position information related to a position and orientation of the pattern on the object, the position information being content of the reference information encoded in the pattern or extracted from a data base of a data storage device as part of an object data set comprising reference information of the object and being assignable to the object by the reference information encoded in the pattern.

12. The automated machine according to claim 11, wherein:

the point of interest is a measurement point of an object to be measured,

the automated machine is a coordinate measuring machine for determining at least one spatial coordinate of the measurement point,

the drive mechanism is adapted to drive a probe head in a manner such that the probe head is capable to move relative to the base for approaching the measurement point, and

the probe head is attached to the frame structure.

13. The automated machine according to claim 11, wherein:

the at least one camera is adapted to identify the pattern provided on the object by means of a one- or two-dimensional pattern, and is adapted to identify the pattern given in one of the following forms;

printed on the object or on a sticker that is applied on the object, or

engraved on the object.

14. The automated machine according to claim 11, wherein:

the reference information included in the pattern comprises object-related information, information about the object type, a serial number or dimensions of the object, and

the least one camera is adapted to read the object-related information from the identified pattern present in the at least one image.

15. The automated machine according to claim 11, wherein:

the object-related information is encoded in the pattern and its respective reference information, respectively, and

the at least one camera is adapted to decode the pattern and reference information, respectively.

16. The automated machine according to claim 15, further comprising:

the data storage device that is adapted to store and to provide object data comprising data sets of a plurality of objects, each data set comprising the reference information of the object and position information related to a position and orientation of the pattern on the object, wherein the controller is adapted to determine the position and orientation of the object based on the position and orientation of the pattern relative to the at least one camera and on the position information provided from the storage and related to the position and orientation of the pattern on the object.

17. The automated machine according to claim 11, wherein:

the reference information of the pattern comprises position information related to a position and orientation of the pattern on the object;

the at least one camera is adapted to read the position information from the identified pattern comprising the reference information; and

the controller is adapted to determine the position and orientation of the object based on the position information provided by the pattern and relative to position and orientation of the pattern on the object and on the determined position of the pattern relative to the at least one camera.

18. The automated machine according to claim 11, wherein

the at least one camera is adapted to determine a distortion of the pattern and to determine the position and orientation of the pattern based on the determined distortion, wherein the controller is adapted to determine the position and orientation of the object in six degrees of freedom.

19. A non-transitory computer program product, comprising program code which is stored on a machine-readable medium, and having computer-executable instructions for performing:

capturing at least one image of the object by means of at least one camera of the automated machine,

identifying the pattern in the at least one image,

determining before starting the measurement a position and orientation of the pattern relative to the at least one camera by image recognition,

determining a position and orientation of the object based on the determined position and orientation of the pattern relative to the camera and on provided position information related to a position and orientation of the pattern on the object, and

controlling the drive mechanism based on the determined position and orientation of the object,

wherein the identifying the pattern comprises reading position information related to a position and orientation of the pattern on the object, the position information being content of the reference information encoded in the pattern or is extracted from a data base of a data storage device as part of an object data set comprising reference information of the object and being assignable to the object by the reference information encoded in the pattern.