Object detection with distributed sensor array

• US 9,269,152 B1
• Filed: 09/07/2011
• Issued: 02/23/2016
• Est. Priority Date: 09/07/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Worley, William Spencer III, Shepelev, Petr G., Crump, Edward Dietz, Sjoberg, Robert W.
• Primary Examiner(s): Shah, Kamini S
• Assistant Examiner(s): Hann, Jay B
• Application Number:
  US13/226,990
• Time in Patent Office: 1,630 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Objects within a scene are modeled in two- or three-dimensions by acquiring slices of data from a distributed sensor array and generating the model of the object at least in part from those slices. The distributed sensor array may comprise optical transmitters and optical receivers configured such that they may be individually addressed and activated. The system described herein may be used to support an augmented reality environment.

Claims

1. A system comprising:

a processor;

a distributed sensor array coupled to the processor and comprising;

(i) multiple optical transmitters, each optical transmitter of the multiple optical transmitters configured to generate an optical signal, and (ii) multiple non-imaging optical receivers;

memory, accessible by the processor;

a spatial analysis module stored in the memory and configured to;

selectively activate a subset of the multiple optical transmitters of the distributed sensor array to cause the subset of the multiple optical transmitters to generate a corresponding optical signal;

receive, by at least a subset of the multiple non-imaging optical receivers, an intensity of the optical signal provided by the subset of the multiple optical transmitters to form a slice;

selectively activate different corresponding subsets of the multiple optical transmitters to generate different corresponding signals and receive, by different corresponding subsets of the multiple non-imaging optical receivers, intensities of the different corresponding signals to acquire multiple respective slices;

generate a model of at least a portion of an exterior of an object based at least in part on the multiple respective slices; and

utilizing information associated with the object to cause an adjustment of one or more characteristics of the distributed sensor array, the adjustment including adjusting a scanning of at least one optical transmitter of the multiple optical transmitters to reduce occlusion associated with the object.

2. The system of claim 1, wherein the generating of the model comprises processing the multiple respective slices with a projection-slice theorem.

3. The system of claim 1, wherein the generating of the model comprises processing the multiple respective slices with a visual hull technique.

4. The system of claim 1, wherein the model comprises a three-dimensional model of the at least a portion of the exterior of the object.

5. The system of claim 1, wherein the subset of the multiple optical transmitters comprises a single optical transmitter.

6. The system of claim 1, wherein the multiple optical transmitters and the multiple non-imaging optical receivers operate in a non-visible light spectral band.

7. The system of claim 1, wherein the multiple optical transmitters and the multiple non-imaging optical receivers operate in a visible light spectral band.

8. The system of claim 1, wherein the multiple optical transmitters and the multiple non-imaging optical receivers operate in a spectral band which is substantially absorbed, substantially reflected, or both from the object.

9. The system of claim 1, wherein the intensity of the optical signal is affected by interference or diffraction of light by the object.

10. The system of claim 1, wherein the multiple optical transmitters and the multiple non-imaging optical receivers are configured to transmit and receive signals, respectively, with a wide-angle field of view of at least four steradians.

11. The system of claim 1, wherein the distributed sensor array further comprises one or more mirrors configured to direct at least a portion of the optical signals.

12. The system of claim 1, wherein the distributed sensor array is emplaced such that a first portion of the array is visible to a second portion of the distributed sensor array.

13. The system of claim 1, further comprising a calibration module stored in the memory and configured to determine relative positions of the multiple optical transmitters and the multiple non-imaging optical receivers in the distributed sensor array.

14. The system of claim 1, wherein the multiple optical transmitters comprise light fixtures configured to also provide general illumination.

15. The system of claim 1, wherein at least a portion of the distributed sensor array is configured to move during selective activation of the optical transmitters.

16. The system of claim 1, further comprising a sensor configured to detect an object and wherein the spatial analysis module is further configured to alter the selective activation of the subset of the multiple optical transmitters of the distributed sensor array at least partly in response to the detection.

17. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

selectively activating an optical transmitter in a distributed sensor array that includes a plurality of optical transmitters and a plurality of non-imaging optical receivers;

receiving at the plurality of non-imaging optical receivers a signal from the optical transmitter, wherein the signal is modified by interaction with an object or unmodified after bypassing the object, to form a slice;

selectively activating different optical transmitters of the plurality of optical transmitters and receiving a different signal from a different non-imaging optical receiver of the plurality of non-imaging optical receivers to acquire multiple different slices;

generating a model of the object based at least in part on the multiple different slices;

determining a region of interest associated with the object; and

adjusting a scanning of at least one optical transmitter in the distributed sensor array to reduce occlusion within the region of interest by the object.

18. The one or more non-transitory computer-readable storage media of claim 17, wherein the generating the model comprises processing the multiple different slices with a projection-slice theorem.

19. The one or more non-transitory computer-readable storage media of claim 17, wherein the optical transmitter and the plurality of non-imaging optical receivers are configured to operate in a visible-light spectral band.

20. The one or more non-transitory computer-readable storage media of claim 19, wherein the plurality of optical transmitters are configured to be illuminated more than 120 times per second.

21. The one or more non-transitory computer-readable storage media of claim 17, the acts further comprising supplementing the model of the object with structured light data.

22. The one or more non-transitory computer-readable storage media of claim 17, further comprising selectively activating a light fixture to generate the optical signal for reception at the plurality of non-imaging optical receivers.

23. The one or more non-transitory computer-readable storage media of claim 17, further comprising directing a plurality of dynamic mirrors to direct at least a portion of the optical signal to one or more of the plurality of non-imaging optical receivers.

24. A method comprising:

detecting an object within an environment;

selectively activating an optical transmitter of one or more optical transmitters in a distributed sensor array that includes the one or more optical transmitters and one or more non-imaging optical receivers;

causing the optical transmitter to generate a signal that is directed to a location within the environment that is associated with the object;

receiving, at an non-imaging optical receiver of the one or more non-imaging optical receivers, the signal, the signal being modified by interaction with the object or unmodified after bypassing the object, to form a slice;

determining information associated with the object; and

causing an adjustment of one or more characteristics of the distributed sensor array based at least partly on the information associated with the object, the adjustment including adjusting a scanning of the optical transmitter to reduce occlusion associated with the object.

25. The method as recited in claim 24, further comprising:

selectively activating a different optical transmitter of the one or more optical transmitters to generate a different signal; and

receiving, by the different optical transmitter of the one or more optical transmitters, the different signal to acquire multiple different slices.

26. The method as recited in claim 25, further comprising generating a model of the object based at least in part on the multiple different slices.

27. A system comprising:

a processor;

a distributed sensor array coupled to the processor and comprising;

(i) multiple optical transmitters that are configured to generate an optical signal, and (ii) multiple non-imaging optical receivers;

memory, accessible by the processor;

a spatial analysis module stored in the memory and configured to;

detect an object within the system;

selectively activate an optical transmitter of the multiple optical transmitters to cause the optical transmitter to direct a signal towards a location associated with the object;

receive the signal to form a slice;

selectively activate a different optical transmitter of the multiple optical transmitters and receive a different signal to acquire multiple respective slices; and

causing an adjustment of one or more characteristics of the distributed sensor array based at least partly on information associated with the object, the adjustment including adjusting a scanning of the optical transmitter to reduce occlusion associated with the object.

28. The system as recited in claim 27, wherein the spatial analysis module is further configured to generate a model of at least a portion of an exterior of the object based at least in part on the multiple respective slices.

29. The system as recited in claim 27, wherein the model includes a two-dimensional model or a three-dimensional model of the at least the portion of the exterior of the object.

30. The system as recited in claim 27, wherein the causing the adjustment of the one or more characteristics of the distributed sensor array includes causing an increase of the scanning of the optical transmitter based at least partly on information associated with the object.

31. The system as recited in claim 27, wherein the causing the adjustment of the one or more characteristics of the distributed sensory array includes causing an increase in resolution associated with the optical transmitter.