Method And Apparatus For Object Pose Estimation

• US 20130051626A1
• Filed: 08/30/2011
• Published: 02/28/2013
• Est. Priority Date: 08/30/2011
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Fu, Guoyi, Abadpour, Arash, Moravec, Ivo
• Granted Patent: US 8,467,596 B2
• Time in Patent Office: Days
• US Class Current: 382/106
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects

Abstract

A pose of an object is estimated from an from an input image and an object pose estimation is then stored by: inputting an image containing an object; creating a binary mask of the input image; extracting a set of singlets from the binary mask of the input image, each singlet representing points in an inner and outer contour of the object in the input image; connecting the set of singlets into a mesh represented as a duplex matrix; comparing two duplex matrices to produce a set of candidate poses; and producing an object pose estimate, and storing the object pose estimate. The estimated pose of the object is refined by: inputting an image of an object in an estimated pose, a model of the object, and parameters of a camera used to take the image of the object in the estimated pose; projecting the model of the object into a virtual image of the object using the parameters of the camera and initial pose parameters to obtain a binary mask image and image depth information; and updating the initial pose parameters to new pose parameters using the binary mask image and image depth information and updating the new pose parameters iteratively to minimize an energy function or until a maximum number of iterations is reached.

Claims

1. A method for estimating a pose of an object from an input image and storing an object pose estimation, comprising:

inputting an image containing an object;

creating a binary mask of the input image;

extracting a set of singlets from the binary mask of the input image, each singlet representing points in an inner and outer contour of the object in the input image;

connecting the set of singlets into a mesh represented as a duplex matrix;

comparing two duplex matrices to produce a set of candidate poses; and

producing an object pose estimate, and storing the object pose estimate.

2. A method for estimating a pose of an object as in claim 1 further comprising inputting a plurality of images of the object, each image containing a view of the object that is different than a view in each other of the plurality of images.

3. A method for estimating a pose of an object as in claim 2 further comprising generating the view in each of the plurality of images of the object using a CAD model for the object.

4. A method for estimating a pose of an object as in claim 2 further comprising generating the view in each of the plurality of images of the object using a robot equipped with a camera.

5. A method for estimating a pose of an object as in claim 1 further comprising detecting the object in the input image and calculating a bounding box of the object.

6. A method for estimating a pose of an object as in claim 1 further comprising extracting inner and outer contours of the object.

7. A method for estimating a pose of an object as in claim 1 further comprising refining the object pose estimate by modeling the pose estimate as an optimization of an energy function.

8. A method for estimating a pose of an object as in claim 7 further comprising calculating pose estimates iteratively to minimize an energy value in the energy function.

9. A method for estimating a pose of an object as in claim 8 further comprising calculating a velocity screw to iteratively calculate pose estimates.

10. A method for estimating a pose of an object as in claim 7 further comprising projecting a model contour curve of the object into a virtual image using an rendering application programming interface (API).

20. One or more tangible, non-transitory computer-readable media having computer-readable instructions thereon, which, when executed by a processor, estimates a pose of an object from an input image according to the method of claim 1.

11. An apparatus for estimating a pose of an object from an input image and storing an object pose estimation, comprising:

a processor that

receives an input image containing an object;

creates a binary mask of the input image;

extracts a set of singlets from the binary mask of the input image, each singlet representing points in an inner and outer contour of the object in the input image;

connects the set of singlets into a mesh represented as a duplex matrix;

compares two duplex matrices to produce a set of candidate poses; and

produces an object pose estimate; and

a memory that stores the object pose estimate.

12. An apparatus for estimating a pose of an object as in claim 11 wherein the processor receives a plurality of images of the object, each image containing a view of the object that is different than a view in each other of the plurality of images.

13. An apparatus for estimating a pose of an object as in claim 12 further comprising a storage that stores a CAD model for the object and the processor generates the view in each of the plurality of images of the object using the CAD model for the object.

14. An apparatus for estimating a pose of an object as in claim 12 further comprising a robot equipped with a camera that generates the view in each of the plurality of images of the object.

15. A method for refining an estimated pose of an object, comprising:

inputting an image of an object in an estimated pose, a model of the object, and parameters of a camera used to take the image of the object in the estimated pose;

projecting the model of the object into a virtual image of the object using the parameters of the camera and initial pose parameters to obtain a binary mask image and image depth information;

updating the initial pose parameters to new pose parameters using the binary mask image and image depth information and updating the new pose parameters iteratively to minimize an energy function or until a maximum number of iterations is reached.

16. A method for refining an estimated pose of an object as in claim 15, further comprising:

calculating a mean value inside and outside the object using a cost function; and

calculating a contour C of the object from the binary mask image and 3D contour points using the image depth information.

17. A method for refining an estimated pose of an object as in claim 16, further comprising:

calculating a gradient ∇

T(x,y) in x and y direction for the contour points;

calculating the image Jacobian matrix; and

calculating a gradient flow and velocity screw using a Gaussian Newton method.

18. A method for refining an estimated pose of an object as in claim 17, further comprising:

calculating a relative displacement using the velocity screw and a given stepwise.

19. A method for refining an estimated pose of an object as in claim 15, wherein the image of an object in an estimated pose is formed by:

inputting an image containing an object;

creating a binary mask of the input image;

extracting a set of singlets from the binary mask of the input image, each singlet representing points in an inner and outer contour of the object in the input image;

connecting the set of singlets into a mesh represented as a duplex matrix;

comparing two duplex matrices to produce a set of candidate poses; and

producing an object pose estimate.