System and method for finding correspondence between cameras in a three-dimensional vision system

• US 8,600,192 B2
• Filed: 12/08/2010
• Issued: 12/03/2013
• Est. Priority Date: 12/08/2010
• Status: Active Grant

• Current Assignee: Cognex Corporation
• Original Assignee: Cognex Corporation
• Inventors: Liu, Lifeng, Wallack, Aaron S., Marrion, Cyril C.
• Primary Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US12/962,918
• Publication Number:
  US 20120148145A1
• Time in Patent Office: 1,091 Days
• Field of Search: 382/154, 382/190, 382/285, 382/291, 382/295, 382/296, 382/305, 382/312, 348/47, 348/207.99, 345/419, 345/653, 345/654, 345/662, 700/245, 700/259
• US Class Current: 382/285
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19

Abstract

This invention provides a system and method for determining correspondence between camera assemblies in a 3D vision system implementation having a plurality of cameras arranged at different orientations with respect to a scene, so as to acquire contemporaneous images of a runtime object and determine the pose of the object, and in which at least one of the camera assemblies includes a non-perspective lens. The searched 2D object features of the acquired non-perspective image, corresponding to trained object features in the non-perspective camera assembly, can be combined with the searched 2D object features in images of other camera assemblies (perspective or non-perspective), based on their trained object features to generate a set of 3D image features and thereby determine a 3D pose of the object. In this manner the speed and accuracy of the overall pose determination process is improved. The non-perspective lens can be a telecentric lens.

Claims

1. A vision system that determines 3D pose of an object comprising:

a plurality of cameras in which a lens of at least one of the cameras is a non-perspective lens to acquire a non-perspective image, the cameras being respectively oriented with a discrete field of view so as to contemporaneously acquire a plurality of images of the object, respectively, at a known position with respect to the field of view, the lens of the at least one of the cameras being mounted on the camera, and the camera including an image sensor and an onboard vision processor;

a search process, operatively connected to the cameras, that searches for 2D feature points in at least some of the acquired images, including the non-perspective image;

an association process that associates the 2D feature points with known 3D model points; and

a pose determination process that computes the 3D pose based upon the 2D feature points and known 3D model points.

2. The vision system as set forth in claim 1 wherein the cameras are non-perspective cameras with respective non-perspective lenses.

3. The vision system as set forth in claim 2 wherein the lens for at least one of the cameras is a telecentric lens.

4. The vision system as set forth in claim 2 wherein the lens for a first camera of the plurality of cameras is a non-perspective lens and the lens for a second camera of the plurality of cameras is a non-perspective lens, and further comprising a trained model for use by the second camera to search for the 2D features, the trained model being based upon an affine transform of a first model with at least one reference point in the image acquired by the first camera.

5. The vision system as set forth in claim 4 wherein the affine transform is based upon at least a subset of intrinsics and extrinsics of the first camera and the second camera.

6. The vision system as set forth in claim 5 wherein the search process is constructed and arranged to acquire a second image with the second camera and to search for 2D features in the second image within a predetermined range based upon the trained model.

7. The vision system as set forth in claim 1 wherein the 3D pose is provided as data to at least one of a device and a procedure that performs an operation upon the object.

8. The vision system as set forth in claim 7 wherein the device is a robot manipulator that moves with respect to the object.

9. The vision system as set forth in claim 1 wherein the pose determination process includes a triangulation process based upon light rays extending between respective of the cameras and feature points on the object.

10. The vision system as set forth in claim 9 wherein the triangulation process is constructed and arranged to employ least squares fit to determine points of intersection of the light rays.

11. A method for determining 3D pose of an object comprising the steps of:

locating a plurality of cameras wherein a lens of at least one of the cameras is a non-perspective lens, the cameras being respectively oriented with a discrete field of view so as to contemporaneously acquire a plurality of images of the object at a known position with respect to the field of view, including at least one non-perspective image, the lens of the at least one of the cameras being mounted on the camera, and the camera including an image sensor and an onboard vision processor;

searching for 2D feature points in at least some of the images of the plurality of images including the non-perspective image;

associating the 2D feature points with known 3D model points; and

computing the 3D pose based upon the 2D feature points and known 3D model points.

12. The method as set forth in claim 11 wherein the images searched in the step of searching are non-perspective images acquired with by the cameras with respective non-perspective lenses.

13. The method as set forth in claim 12 wherein at least one of the non-perspective lenses is a telecentric lens.

14. The method as set forth in claim 12 wherein the lens for a first camera of the plurality of cameras is a non-perspective lens and the lens for a second camera of the plurality of cameras is a non-perspective lens, and further comprising searching for 2D features with the second camera using a trained model, the trained model being based upon an affine transform of a first model with at least one reference point in the image acquired by the first camera.

15. The method as set forth in claim 13 further comprising computing the affine transform based upon at least a subset of intrinsics and extrinsics of the first camera and the second camera.

16. The method as set forth in claim 15 further comprising acquiring a second image with the second camera and searching for 2D features in the second image within a predetermined range based upon the trained model.

17. The method as set forth in claim 11 further comprising performing an operation on the object based upon the 3D pose.

18. The method as set forth in claim 17 wherein the step of performing the operation includes moving a robot manipulator with respect to the object.

19. A non-transitory recorded computer readable medium operating on vision system that determines 3D pose of an object imaged by a plurality of cameras, in which the lens of at least one of the cameras is a non-perspective lens to acquire a non-perspective image, the cameras respectively being oriented with a discrete field of view so as to contemporaneously acquire a plurality of images of the object at a known position with respect to the field of view, the lens of the at least one of the cameras being mounted on the camera, and the camera including an image sensor and an onboard vision processor, including program instructions for performing the steps of:

searching for 2D feature points in at least some of the plurality of images including the non-perspective image;

associating the 2D feature points with known 3D model points; and

computing the 3D pose based upon the 2D feature points and known 3D model points.

20. The non-transitory computer readable medium as set forth in claim 19 wherein the step of computing includes performing triangulation upon light rays extending between respective of the plurality of cameras and feature points on the object.

21. The non-transitory computer readable medium as set forth in claim 20 wherein the step of searching includes operating a search tool that locates features located on flat surfaces on the object based upon trained models of the features.

22. The non-transitory computer readable medium as set forth in claim 21 wherein the trained models of the features are located based upon at least a subset of intrinsics and extrinsics of respective of the plurality of cameras.

23. The non-transitory computer readable medium as set forth in claim 19 wherein the 3D pose is employed to perform an operation on the object.