Projection block extraction

• US 9,241,141 B1
• Filed: 12/23/2011
• Issued: 01/19/2016
• Est. Priority Date: 12/23/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Chang, Samuel Henry, Coley, Christopher, Shepelev, Petr G., Gray, Lynnda
• Primary Examiner(s): CHIO, TAT CHI
• Application Number:
  US13/336,458
• Time in Patent Office: 1,488 Days
• US Class Current: 1/1
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

Sensing systems using projection blocks of structured light provide useful information about the surfaces they are projected onto. A camera captures an image which contains a captured block, that is, an image of the projection block interacting with the surface. Corrections may be applied to the captured image to compensate for distortions produced by projector and/or camera systems and produce a corrected image. Noise may be removed from the corrected image using a bilateral filter, producing a filtered image. The filtered image may be processed to determine a boundary of the captured block.

Claims

1. A system comprising:

one or more processors;

a projector coupled to the one or more processors and configured to project a projection block comprising a structured light pattern onto a surface of an object within an environment;

a camera coupled to the one or more processors and configured to capture an image of the object, the captured image containing a captured block that comprises the projection block as projected onto the surface of the object; and

one or more computer-readable storage media storing computer-executable instructions that, when executed on the one or more processors, cause the one or more processors to perform acts comprising;

applying distortion correction to at least a portion of the captured image to generate a corrected image, wherein applying the distortion correction compensates for a distortion factor associated with characteristics of the projector;

applying a bilateral filter to at least a portion of the corrected image to form a filtered image; and

determining a boundary of the captured block in the filtered image.

2. The system of claim 1, wherein the distortion correction compensates for distortion in the camera when capturing the image.

3. The system of claim 1, the determining of the boundary comprising:

for at least one of a row or column of pixels in the filtered image, determining one or more peaks of intensity associated with the structured light pattern of the projection block, wherein the peaks are pixels having one or more characteristics above a threshold; and

designating the one or more determined peaks at a beginning and end of the row or column as points along the boundary.

4. The system of claim 1, the acts further comprising determining a mapping based at least in part on mapping one or more corners of the captured block to corresponding corners of the projection block.

5. The system of claim 4, the acts further comprising applying the mapping to the captured block to form a corrected block.

6. The system of claim 1, wherein the applying of the bilateral filter comprises replacing an intensity value of a pixel with a weighted average of intensity values from other pixels within a pre-determined distance of the pixel.

7. The system of claim 1, wherein the structured light pattern of the projection block comprises a regular periodic arrangement of pixels having at least two intensities.

8. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

receiving a captured image that includes a captured block, the captured block comprising a projection block after interaction with a surface;

applying one or more corrections to the captured image, wherein applying the one or more corrections compensates for distortion associated with characteristics of a projector that presents the projection block on the surface;

applying a bilateral filter to at least a portion of the captured image to form a filtered image; and

determining at least a portion of a boundary of the captured block within the filtered image.

9. The one or more computer-readable storage media of claim 8, wherein the entire captured block is present within the captured image.

10. The one or more computer-readable storage media of claim 8, wherein the projection block comprises a structured light pattern that has been projected onto the surface.

11. The one or more computer-readable storage media of claim 8, wherein the one or more corrections compensate for distortion associated with characteristics of a camera that receives the captured image.

12. The one or more computer-readable storage media of claim 8, the acts further comprising:

mapping points on at least a portion of the boundary of the captured block to corresponding corners of the projection block; and

determining a transform based at least in part on the mapping.

13. The one or more computer-readable storage media of claim 12, the acts further comprising applying the transform to the captured block to form a corrected block.

14. The one or more computer-readable storage media of claim 8, wherein the determining at least a portion of the boundary of the captured block within the filtered image comprises:

identifying a structured light pattern used to generate the projection block;

for each row or column of pixels within at least a portion of the filtered image, determining pixels corresponding to transitions between two or more intensity states in the structured light pattern; and

designating the determined pixels at the beginning and end of the row or column as points on the boundary of the captured block.

15. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

accessing a captured image comprising a structured light pattern interacting with a surface;

applying one or more corrections to the captured image, wherein applying the one or more corrections compensates for distortion associated with characteristics of a projector that presents the structured light pattern on the surface;

applying a bilateral filter to at least a portion of the captured image to form a filtered image; and

determining at least a portion of a boundary of the structured light pattern within the filtered image.

16. The one or more computer-readable storage media of claim 15, wherein the bilateral filter is configured to be proportionate to a feature size of the structured light pattern.

17. The one or more computer-readable storage media of claim 15, the acts further comprising mapping points of the structured light pattern in the captured image to corresponding points in the structured light pattern prior to projection.

18. The one or more computer-readable storage media of claim 15, wherein the structured light pattern is imperceptible to a human when projected.