Single source multi camera vision system

• US 10,514,256 B1
• Filed: 05/06/2013
• Issued: 12/24/2019
• Est. Priority Date: 05/06/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Kamarshi, Vijay, Sjoberg, Robert Warren, Haskin, Menashe, Tikare, Amit
• Primary Examiner(s): Lubit, Ryan A
• Application Number:
  US13/888,047
• Time in Patent Office: 2,423 Days
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors

Abstract

In some examples, a vision system includes multiple time of flight (ToF) cameras and a single illumination source. The illumination source and the multiple ToF cameras may be synchronized with each other, such as through a phase locked loop based on a generated control signal. A first one of the ToF cameras may be co-located with the illumination source, and a second one of the ToF cameras may be spaced away from the illumination source and the first ToF camera. For instance, the first ToF camera may have a wider field of view (FoV) for generating depth mapping of a scene, while the second ToF camera may have a narrower FoV for generating higher resolution depth mapping of a particular portion of the scene, such as for gesture recognition.

Claims

1. A system comprising:

one or more processors;

a light source to illuminate a scene, the light source including a first emitter and a second emitter;

a first time of flight (ToF) camera including a first lens and a first image sensor, the first lens providing a first field of view;

a second ToF camera including a second lens and a second image sensor, the second lens providing a second field of view that is narrower than the first field of view; and

one or more computer-readable media storing instructions which, when executed by the one or more processors, cause the one or more processors to be configured to;

receive first data from the first image sensor, the first data based at least in part on first light reflected off an object within the first field of view;

determine, based at least in part on the first data, a location of the object; and

orient, based at least in part on the location of the object, the second ToF camera to include the object within the second field of view, the orienting including adjusting a zoom parameter of the second ToF camera.

2. The system as recited in claim 1, the instructions further causing the one or more processors to be configured to modulate the second image sensor at a same frequency used to modulate the light from the light source.

3. The system as recited in claim 1, wherein the first emitter and the second emitter comprise at least one of infrared light emitting diodes, infrared lasers, or infrared laser diodes.

4. A method comprising:

positioning a first time of flight (ToF) camera with a light source in a scene, the first ToF camera including a first image sensor;

positioning a second ToF camera spaced away from the first ToF camera, the second ToF camera including a second image sensor;

receiving, from the first image sensor, first data corresponding to the scene;

identifying, from the first data, an object within the scene;

orienting, based at least in part on the identifying, the second ToF camera to include the object within a field of view of the second ToF camera, the orienting including adjusting a zoom parameter of the second ToF camera; and

receiving, from the second image sensor, second data representing the object at a higher resolution relative to the first data.

5. The method as recited in claim 4, further comprising providing a control signal to the second ToF camera and at least one of the first ToF camera or the light source.

6. The method as recited in claim 5, further comprising executing one or more processors to provide the control signal.

7. The method as recited in claim 4, further comprising:

generating a depth map of a scene within a field of view of the first ToF camera based at least in part on the first data; and

wherein identifying the object is based at least in part on the depth map.

8. The method as recited in claim 4, wherein the object is a hand of a user, the method further comprising

recognizing, based at least in part on the second data, at least one gesture made by the hand;

and

performing, based at least in part on the at least one gesture, an operation.

9. The method as recited in claim 4, further comprising determining, based at least in part on a phase difference between first light emitted by the light source and second light received by the second ToF camera, a distance from the second ToF camera to a surface of the object.

10. The method as recited in claim 4, further comprising determining, based at least in part on the first data and the second data, a distance from a surface of the object to the second ToF camera.

11. The method as recited in claim 4, further comprising determining, based at least in part on echolocation, a distance between the second ToF camera and the object.

12. A system comprising:

a light source including a first emitter and a second emitter, the first emitter and the second emitter emitting non-visible light toward a scene;

a first time of flight (ToF) camera positioned between the first emitter and the second emitter, the first ToF camera configured to receive first light reflected off an object through a first lens corresponding to a first field of view;

a second ToF camera spaced apart from the first ToF camera and the light source, the second ToF camera including a second lens corresponding to a second field of view that is narrower than the first field of view; and

a processor configured by one or more executable instructions to;

recognize, based at least in part on first data received from the first ToF camera, the object within the scene;

orient, based at least in part on recognizing the object within the scene, the second ToF camera to include the object within the second field of view, the orienting including adjusting a zoom parameter of the second ToF camera; and

receive, based at least in part on orienting the second ToF camera, second data representing the object at a higher resolution relative to the first data.

13. The system as recited in claim 12, wherein the processor is further configured generate, based at least in part on the first data and the second data, a depth map of the scene.

14. The system as recited in claim 12, wherein the second data is based at least in part on a phase difference between the non-visible light emitted by the light source and second light received by the second ToF camera.

15. The system as recited in claim 12, wherein adjusting the zoom parameter of the second ToF camera comprises at least adjusting an optical zoom of the second ToF camera to zoom in on the object.

16. The system as recited in claim 12, wherein the processor is further configured provide a control signal to the second ToF camera and at least one of the light source or the first ToF camera, the control signal synchronizing a frequency of modulation of the non-visible light emitted by the light source with a frequency of modulation of an image sensor of the second ToF camera.

17. One or more non-transitory computer-readable media maintaining instructions which, when executed by one or more processors, cause the one or more processors to perform acts comprising:

receiving, from a first time of flight (ToF) camera, first image sensor data corresponding to first light reflected off an object within a scene;

identifying, from the first image sensor data, a location of the object within the scene and a first distance from the first ToF camera to a surface of the object;

orienting, based at least in part on the identifying, the second ToF camera to include the object within a field of view of the second ToF camera, the orienting including adjusting a zoom parameter of the second ToF camera;

receiving, from the second ToF camera, second image sensor data corresponding to second light reflected off the object, the second image sensor data representing the object at a higher resolution relative to the first image sensor data; and

determining, based at least in part on the second image sensor data and the first distance, a second distance from the second ToF camera to the surface of the object.

18. The one or more non-transitory computer-readable media as recited in claim 17, wherein the second image sensor data is based at least in part on a phase difference between light emitted by a light source and the second light.

19. The one or more non-transitory computer-readable media as recited in claim 17, the acts further comprising providing a control signal to the second ToF camera and the first ToF camera, the control signal synchronizing a frequency of modulation of light emitted by a light source with a frequency of modulation of an image sensor of the second ToF camera.