Characterization of a scene with structured light

• US 8,845,107 B1
• Filed: 12/23/2010
• Issued: 09/30/2014
• Est. Priority Date: 12/23/2010
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Rawles, LLC (Amazon.com, Inc.)
• Inventors: Coley, Christopher, Worley, William Spencer III, Betts-LaCroix, Jonathan
• Primary Examiner(s): CHOWDHURY, SULTAN U.
• Application Number:
  US12/977,924
• Time in Patent Office: 1,377 Days
• Field of Search: 353/28, 353/30, 353/39, 353 69- 70, 353/79, 353/122, 345/420, 345/418, 345/419, 345/423, 345/441, 345/442, 348/40, 348/42, 348/51, 348/54, 348/58, 348/59, 348/135, 348743-747, 380/232, 380/236
• US Class Current: 353/28
• CPC Class Codes: G01B 11/022 by means of tv-camera scanning
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G03B 21/147 Optical correction of image distortions, e.g. keystone
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/012 Dimensioning, tolerancing
• CPC Class Codes: G06T 2219/2004 Aligning objects, relative positioning of parts
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Objects and their placement within a particular scene may be characterized using structured light with no or minimal manual intervention or calibration to the scene. The structured light pattern may be rendered with visible light, yet imperceptible to users, by generating a set of opposing excursions in one or more image properties for durations which are shorter than the integration time of the human eye. A set of assumptions and ongoing testing of those assumptions against physical objects identified within the environment allows characterization of the scene and the objects therein.

Claims

1. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

determining a topology of a scene;

identifying one or more objects within the scene;

determining a dimension of objects within the scene based at least in part upon testing one or more assumptions of a preliminary dimension of the one or more identified objects within the scene; and

generating a model of the scene based at least in part upon the determined topology, the identified objects, and the determined dimension.

2. The one or more non-transitory computer-readable storage media of claim 1, wherein the determining the topology of the scene comprises scanning the scene with a structured light pattern.

3. The one or more non-transitory computer-readable storage media of claim 2, wherein the structured light pattern comprises a pseudorandom noise pattern modulating a plurality of visible light sub-frames with a set of opposing excursions in one or more image properties.

4. The one or more non-transitory computer-readable storage media of claim 3, wherein the excursions in one or more image properties vary dynamically between pluralities of visible light sub-frames.

5. The one or more non-transitory computer-readable storage media of claim 3, wherein the pseudorandom noise pattern comprises dynamic pixel blocks.

6. The one or more non-transitory computer-readable storage media of claim 1, wherein the testing the one or more assumptions comprises:

applying the preliminary dimension to the one or more identified objects within the scene based at least in part upon their identification;

applying a tentative dimension to the one or more unidentified objects based at least in part upon the preliminary dimension of the one or more identified object and a relative displacement of the one or more unidentified objects compared to the one or more identified objects;

testing the tentative dimension of the object and discarding contradictory dimensional results; and

confirming the preliminary dimension and the tentative dimension as actual dimensions when error associated with the preliminary dimension, tentative dimension, or both is reduced below a pre-determined error threshold.

7. The one or more non-transitory computer-readable storage media of claim 2, further comprising retrieving from an object datastore dimensional data about the one or more identified objects and using the dimensional data to determine the preliminary dimension.

8. The one or more non-transitory computer-readable storage media of claim 1, wherein the preliminary dimension is based at least in part upon one or more system parameters and one or more object parameters.

9. The one or more non-transitory computer-readable storage media of claim 8, wherein the system parameters comprise state information about one or more components within a system determining the topology of the scene.

10. The one or more non-transitory computer-readable storage media of claim 8, wherein the object parameters comprise state information about the one or more objects within the scene.

12. The one or more non-transitory computer-readable storage media of claim 1, wherein the testing the one or more assumptions comprises:

applying the preliminary dimension to the one or more dimensioned objects within the scene based at least in part upon the determined topology of the scene;

applying a tentative dimension to one or more undimensioned objects based at least in part upon the preliminary dimension of the one or more dimensioned objects and a relative displacement of the undimensioned objects compared to the one or more dimensioned objects;

testing the preliminary dimension of the one or more dimensioned objects and the tentative dimension of the one or more undimensioned objects and discarding contradictory dimensional results; and

confirming the preliminary dimension and the tentative dimension as actual dimensions when error associated with the preliminary dimension, tentative dimension, or both is reduced below a pre-determined error threshold.

11. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

determining a topology of a scene;

locating one or more objects within the scene;

determining a dimension of objects within the scene based at least in part upon testing one or more assumptions of preliminary dimension of the one or more objects within the scene; and

generating a model of the scene based at least in part upon the determined topology and the determined dimension.

13. The one or more non-transitory computer-readable storage media of claim 11, wherein the preliminary dimension is based at least in part upon the comparison of two or more images acquired with differing system parameters.

14. A method comprising:

determining a topology of a scene;

identifying one or more objects within the scene;

determining a dimension of objects within the scene based at least in part upon testing one or more assumptions of a preliminary dimension of the one or more identified objects within the scene; and

generating a model of the scene based at least in part upon the determined topology, the identified objects, and the determined dimension.

15. The method of claim 14, wherein the determining the topology of the scene comprises scanning the scene with a structured light pattern.

16. The method of claim 15, wherein the structured light pattern comprises a pseudorandom noise pattern modulating a plurality of visible light sub-frames with a set of opposing excursions in one or more image properties.

17. The method of claim 16, wherein the excursions in one or more image properties vary dynamically between pluralities of visible light sub-frames.

18. The method of claim 16, wherein the pseudorandom noise pattern comprises dynamic pixel blocks.

19. The method of claim 14, wherein the testing the one or more assumptions comprises:

applying the preliminary dimension to the one or more identified objects within the scene based at least in part upon their identification;

applying a tentative dimension to the one or more unidentified objects based at least in part upon the preliminary dimension of the one or more identified object and a relative displacement of the one or more unidentified objects compared to the one or more identified objects;

testing the tentative dimension of the object and discarding contradictory dimensional results; and

confirming the preliminary dimension and the tentative dimension as actual dimensions when error associated with the preliminary dimension, tentative dimension, or both is reduced below a pre-determined error threshold.

20. The method of claim 15, further comprising retrieving from an object datastore dimensional data about the one or more identified objects and using the dimensional data to determine the preliminary dimension.

21. The method of claim 14, wherein the preliminary dimension is based at least in part upon one or more system parameters and one or more object parameters.

22. The method of claim 21, wherein the system parameters comprise state information about one or more components within a system determining the topology of the scene.

23. The method of claim 21, wherein the object parameters comprise state information about the one or more objects within the scene.

24. A method comprising:

determining a topology of a scene;

locating one or more objects within the scene;

determining a dimension of objects within the scene based at least in part upon testing one or more assumptions of preliminary dimension of the one or more objects within the scene; and

generating a model of the scene based at least in part upon the determined topology and the determined dimension.

25. The method of claim 24, wherein the testing the one or more assumptions comprises:

applying the preliminary dimension to the one or more dimensioned objects within the scene based at least in part upon the determined topology of the scene;

applying a tentative dimension to one or more undimensioned objects based at least in part upon the preliminary dimension of the one or more dimensioned objects and a relative displacement of the undimensioned objects compared to the one or more dimensioned objects;

testing the preliminary dimension of the one or more dimensioned objects and the tentative dimension of the one or more undimensioned objects and discarding contradictory dimensional results; and

confirming the preliminary dimension and the tentative dimension as actual dimensions when error associated with the preliminary dimension, tentative dimension, or both is reduced below a pre-determined error threshold.

26. The method of claim 24, wherein the preliminary dimension is based at least in part upon the comparison of two or more images acquired with differing system parameters.