Method and System for Locating and Picking Objects Using Active Illumination

• US 20090116728A1
• Filed: 11/07/2007
• Published: 05/07/2009
• Est. Priority Date: 11/07/2007
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Raskar, Ramesh, Agrawal, Amit K.
• Granted Patent: US 7,983,487 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: B22D 11/1265 having auxiliary devices for deburring
• CPC Class Codes: B22D 46/00 Controlling, supervising, not restricted to casting covered by a single main group, e.g. for safety reasons controlling or regulating in general G05
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/75 involving models

Abstract

A method and system determines a 3D pose of an object in a scene. Depth edges are determined from a set of images acquired of a scene including multiple objects while varying illumination in the scene. The depth edges are linked to form contours. The images are segmented into regions according to the contours. An occlusion graph is constructed using the regions. The occlusion graph includes a source node representing an unoccluded region of an unoccluded object in scene. The contour associated with the unoccluded region is compared with a set of silhouettes of the objects, in which each silhouette has a known pose. The known pose of a best matching silhouette is selected as the pose of the unoccluded object.

Claims

1. A method for determining a 3D pose of an object in a scene, comprising:

determining depth edges from a set of images acquired of a scene including a plurality of objects while varying illumination in the scene;

linking the depth edges to form contours;

segmenting the images into regions according to the contours;

constructing an occlusion graph using the regions, in which the occlusion graph includes a source node representing an unoccluded region of an unoccluded object in the scene;

comparing the contour associated with the unoccluded region with a set of silhouettes of the plurality of objects, in which each silhouette has a known pose; and

selecting the known pose of a best;

matching silhouette as the pose of the unoccluded object.

2. The method of claim 1, further comprising:

picking the unoccluded object from the scene according to the pose.

3. The method of claim 1, in which the picking is performed by a robotic arm.

4. The method of claim 1, in which, the plurality of objects are similar in appearance and shape.

5. The method of claim 1, in which the objects have a non-Lambertian surface.

6. The method, of claim 1, in which the varying illumination is from light sources above, below, and left and right of the camera.

7. The method of claim 1, in which the camera is a video camera.

8. The method of claim 3, in which the camera is arranged on the robot arm.

9. The method of claim 1, further comprising:

defining the depth edges as discontinuities between a boundary off the object and a background in the scene; and

defining the shadow edges as discontinuities between a shadow cast by the object and the background.

10. The method of claim 1, in which the plurality of objects include specularities.

11. The method of claim 10, in which the set of images are and further comprising:

determining an intrinsic image for each image I_median from medians of gradient at each pixel in the set of images;

replacing each image I_i by I_i=min(I_i, I_median);

obtaining ratio images I_i/I_median; and

determining the depth edges from the ratio images to minimize the specularities.

12. The method of claim 9, in which linking has constraints that there is a parallel shadow edge for each depth edge, and the depth edge and shadow edge cannot coexist at a same pixel in the set of images.

13. The method of claim 1, in which a directed arc runs from a first node to a second node when a first region represented by the first node casts a shadow on a second region represented by the second node, and the source node does not have an incoming arc edge.

14. The method of claim 13, further comprising:

merging the first node and the second node if there is no arc between the first node and the second node

15. The method of claim 3, in which the silhouettes and known poses are prestored in a memory or database.

16. The method of claim 3, in which the silhouettes and known poses are obtained from computer aided design model of different views of the plurality of objects.

17. The method of claim 1, in which the silhouettes include partial silhouettes.

18. The method of claim 1, in which the set of images are acquired by multiple cameras.

19. A system for determining a pose of an object in a scene, comprising:

a camera configured to acquire a set of images of a scene including a plurality of objects while varying illumination in the scene;

means for determining depth edges from the set of images;

means for linking the depth edges to form contours;

means for segmenting the images into regions according to the contours;

means for constructing an occlusion graph using the regions, in which the occlusion graph includes a source node representing an unoccluded region of an unoccluded object in the scene;

a database storing a set of silhouettes of the plurality of objects, in which each silhouette has a known pose;

means for comparing the depth edges corresponding to subgraphs of the occlusion graph with the set of silhouettes; and

means for selecting the known pose of a best matching silhouette as the pose of the unoccluded object.

20. The system of claim 19, further comprising:

means for picking the unoccluded object from the scene according to the pose.