Pose detection

• US 9,129,375 B1
• Filed: 04/25/2012
• Issued: 09/08/2015
• Est. Priority Date: 04/25/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Rawles, LLC (Amazon.com, Inc.)
• Inventors: Liu, Qiang, Chang, Samuel Henry
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Le, Sarah
• Application Number:
  US13/455,949
• Time in Patent Office: 1,231 Days
• Field of Search: 345/651
• US Class Current: 1/1
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06T 3/14 Transformations for image registration, e.g. adjusting or mapping for alignment of images
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: H04N 7/54 the signals being synchronous H04N21/23602, H04N21/23614, H04N21/2365, H04N21/2368, H04N21/4341, H04N21/4342, H04N21/4347, H04N21/4348 take precedence
• CPC Class Codes: H04N 9/28 Arrangements for convergence or focusing
• CPC Class Codes: H04N 9/31 Projection devices for colour picture display , e.g. using electronic spatial light modulators [ESLM]
• CPC Class Codes: H04N 9/3147 Multi-projection systems displays in general H04N9/12; video walls G06F3/1446, G09G2300/026
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

Techniques are described for determining the pose of an object based on a 3D point set representing the object. The 3D point set is rotated into a 2D coordinate system and a model object contour is aligned with the contour of the rotated point set using an iterative process. The aligned model object contour is then rotated back into the original 3D coordinate system, where its pose is assumed to represent the pose of the object.

Claims

1. A system comprising:

one or more processors;

an imaging sensor;

a projector;

one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;

observing, based at least in part on the image sensor, at least a portion of a display medium;

determining a point set corresponding to the observed display medium with respect to a 3D coordinate system;

rotating the point set into two dimensions of the 3D coordinate system by applying a rotation value to the point set;

identifying an observed contour comprising periphery points of the rotated point set;

aligning a model contour with the observed contour, the model contour comprising periphery points of an expected display medium;

inversely applying the rotation value to the aligned model contour to rotate the model contour into three dimensions of the 3D coordinate system;

evaluating a position and orientation of the display medium based on the aligned and rotated model contour; and

using the projector to project content onto the display medium at the evaluated position and orientation of the display medium.

2. The system of claim 1, wherein the rotation value comprises a rotation matrix.

3. The system of claim 1, the acts further comprising:

determining a surface normal of the observed display medium based at least in part on the point set; and

determining the rotation value based at least in part on the surface normal of the observed display medium.

4. The system of claim 1, the acts further comprising:

determining a surface normal of the observed display medium based at least in part on the point set; and

determining the rotation value as a rotation matrix that rotates the surface normal of the observed display medium into the two dimensions of the 3D coordinate system.

5. The system of claim 1, wherein the aligning comprises minimizing differences between the periphery points of the model contour and the periphery points of the observed contour.

6. The system of claim 1, wherein the aligning comprises:

minimizing differences between the periphery points of the model contour and the periphery points of the observed contour in iterations; and

during successive iterations of the minimizing, ignoring periphery points that are least closely aligned.

7. The system of claim 1, wherein the aligning comprises iterative closest point (ICP) minimization to minimize distances between the periphery points of the model contour and the periphery points of the observed contour in successive iterations, and wherein each iteration of the minimization ignores one or more periphery points that were least closely aligned in a previous iteration of the minimization.

8. The system of claim 1, the acts further comprising estimating alignment of the model contour with the observed contour prior to performing the aligning.

9. The system of claim 1, the acts further comprising estimating alignment of the model contour with the observed contour prior to performing the aligning, wherein the estimating is based at least in part on principal component analysis (PCA).

10. The system of claim 1, the acts further comprising estimating alignment of the model contour with the observed contour prior to performing the aligning, wherein the estimating is based at least in part on averaging the periphery points of the observed contour to determine a center of the observed contour.

11. The system of claim 1, wherein the display medium is rectangular.

12. A method of determining a pose of an object,

comprising:

obtaining a point set representing an observation of at least a portion of the object with respect to a 3D coordinate system;

rotating the point set into two dimensions of the 3D coordinate system by applying a rotation value to the point set;

aligning a model contour with a data contour, the model contour comprising periphery points corresponding to the object, the data contour comprising periphery points of the rotated point set; and

inversely applying the rotation value to the aligned model contour.

13. The method of claim 12, wherein the rotation value comprises a rotation matrix.

14. The method of claim 12, further comprising:

determining a surface normal of the observed display medium based at least in part on the point set; and

determining the rotation value based at least in part on the surface normal of the observed display medium.

15. The method of claim 12, further comprising:

determining a surface normal of the observed display medium based at least in part on the point set; and

determining the rotation value as a rotation matrix that rotates the surface normal of the observed display medium into the two dimensions of the 3D coordinate system.

16. The method of claim 12, wherein the aligning comprises minimizing differences between the periphery points of the model contour and the periphery points of the data contour.

17. The method of claim 12, wherein the aligning comprises:

minimizing differences between the periphery points of the model contour and the periphery points of the data contour in iterations; and

during successive iterations of the minimizing, ignoring periphery points that are least closely aligned.

18. The method of claim 12, wherein the aligning comprises iterative closest point (ICP) minimization to minimize distances between the periphery points of the model contour and the periphery points of the data contour in successive iterations, and wherein each iteration of the minimization ignores one or more periphery points that were least closely aligned in a previous iteration of the minimization.

19. The method of claim 12, further comprising estimating alignment of the model contour with the data contour prior to performing the aligning, wherein the estimating is based at least in part on principal component analysis (PCA).

20. The method of claim 12, further comprising estimating alignment of the model contour with the data contour prior to performing the aligning, wherein the estimating is based at least in part on averaging the periphery points of the data contour to determine a center of the data contour.

21. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

obtaining an observed point set representing at least a portion of an observed object in a 3D coordinate system;

rotating the observed point set into two dimensions of the 3D coordinate system;

creating an observed contour representing periphery points of the rotated observed point set;

creating a model contour representing periphery points of an expected object;

aligning the model contour with the observed contour; and

inversely rotating the aligned model contour into the 3D coordinate system.

22. The one or more non-transitory computer-readable media of claim 21, wherein the 2D coordinate system comprises two dimensions of the 3D coordinate system.

23. The one or more non-transitory computer-readable media of claim 21, wherein the aligning comprises minimizing differences between the periphery points of the model contour and the periphery points of the observed contour.

24. The one or more non-transitory computer-readable media of claim 21, wherein the aligning comprises:

minimizing differences between the periphery points of the model contour and the periphery points of the observed contour in iterations; and

during successive iterations of the minimizing, ignoring periphery points that are least closely aligned.

25. The one or more non-transitory computer-readable media of claim 21, wherein the aligning comprises iterative closest point minimization (ICP) to minimize distances between the periphery points of the model contour and the periphery points of the observed contour in successive iterations, and wherein each iteration of the minimization ignores one or more periphery points that were least closely aligned in a previous iteration of the minimization.

26. The one or more non-transitory computer-readable media of claim 21, the acts further comprising estimating alignment of the model contour with the observed contour prior to performing the aligning, wherein the estimating is based at least in part on principal component analysis (PCA).

27. The one or more non-transitory computer-readable media of claim 21, the acts further comprising estimating alignment of the model contour with the observed contour prior to performing the aligning, wherein the estimating is based at least in part on averaging the periphery points of the observed contour to determine a center of the observed contour.