Using multiple cameras to stitch a consolidated 3D depth map

• US 10,178,370 B2
• Filed: 12/19/2016
• Issued: 01/08/2019
• Est. Priority Date: 12/19/2016
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Shintani, Peter, Gonzalez Hernandez, Jose Omar, Clay, William, Espinosa, Pablo Antonio, Ansfield, Fred, Mohapatra, Bibhudendu, Resch, Keith, Usami, Morio
• Primary Examiner(s): Rao, Anand S
• Application Number:
  US15/383,683
• Publication Number:
  US 20180176530A1
• Time in Patent Office: 750 Days
• Field of Search: 348 42- 61, 348135-139
• CPC Class Codes: G01B 11/00 Measuring arrangements characterised by the use of optical techniques
• CPC Class Codes: G01B 11/245 using a plurality of fixed, simultaneously operating transducers G01B11/2408 - G01B11/2425, G01B11/255 take precedence
• CPC Class Codes: G01B 2210/58 Wireless transmission of information between a sensor or probe and a control or evaluation unit
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Each of plural devices includes a laser emitter and a camera for detecting reflections of laser light emitted by the device, so that plural of the devices can generate their own depth maps showing images within the field of view of their cameras. The resulting 3D depth maps from multiple cameras at potentially arbitrary locations can be aggregated to create a more accurate 3D depth map of the area covered by all the individual cameras. Each device/camera may be assigned a unique identification such as a unique number and a mechanism to identify, either electronically or visually, devices in its field of view. Using this information, the relative locations of the cameras can be calculated for purposes of aggregating the multiple depth maps.

Claims

1. A device comprising:

at least one computer medium that is not a transitory signal and that comprises instructions executable by at least one processor to;

generate, using a first device, at least a first image of second and third devices in a field of view of a camera;

the first image being a first 3D depth map;

receive from at least one of the second and third devices information useful for generating a second 3D depth map including an image of the first device; and

generate an aggregate 3D depth map using the first and second 3D depth maps.

2. The device of claim 1, wherein the instructions are executable to:

exchange device identifying information with the second and third devices.

3. The device of claim 1, wherein the information useful for generating a second 3D depth map includes the device identifying information, and the first and second 3D depth maps are generated by a single camera located in respective first location and second locations.

4. The device of claim 1, wherein the information useful for generating a second 3D depth map includes an image of the first device taken by the second device.

5. The device of claim 1, wherein the instructions are executable to:

receive from at least one of the second and third devices information useful for generating a third 3D depth map including an image of the first device; and

generate an aggregate 3D depth map using the first, second, and third 3D depth maps.

6. The device of claim 1, wherein the instructions are executable to:

determine a distance to a device corresponding to an image in the first 3D depth map at least in part based on a size of the image.

7. The device of claim 1, wherein the instructions are executable to:

alter aspects of images in the first and second 3D depth maps for a reference location to render altered images in the aggregate 3D depth map.

8. An assembly comprising:

plural computerized devices;

each computerized device comprising at least one laser emitter configured to output signals useful for generating at least one three dimensional (3D) depth map;

a first one of the computerized devices being programmed with instructions to;

generate, using the first device, at least a first image of second and third devices in a field of view of a camera;

the first image being a first 3D depth map;

receive from at least one of the second and third devices information useful for generating a second 3D depth map including an image of the first device; and

generate an aggregate 3D depth map using the first and second 3D depth maps.

9. The assembly of claim 8, wherein the instructions are executable to:

exchange device identifying information with the second and third devices.

10. The assembly of claim 8, wherein the information useful for generating a second 3D depth snap includes the device identifying information, and the first and second 3D depth maps are generated by a single camera located in respective first location and second locations.

11. The assembly of claim 8, wherein the information useful for generating a second 3D depth map includes an image of the first device taken by the second device.

12. The assembly of claim 8, wherein the instructions are executable to:

receive from at least one of the second and third devices information useful for generating a third 3D depth map including an image of the first device; and

generate an aggregate 3D depth map using the first, second, and third 3D depth maps.

13. The assembly of claim 8, wherein the instructions are executable to:

determine a distance to a device corresponding to an image in the first 3D depth map at least in part based on a size of the image.

14. The assembly of claim 8, wherein the instructions are executable to:

alter aspects of images in the first and second 3D depth maps for a reference location to render altered images in the aggregate 3D depth map.

15. A method comprising:

generating, using a first device, at least a first image of second and third devices in a field of view of a camera;

the first image being a first 3D depth map;

receiving from at least one of the second and third devices information useful for generating a second 3D depth map including an image of the first device; and

generating an aggregate 3D depth map using the first and second 3D depth maps.

16. The method of claim 15, comprising:

exchanging device identifying information with the second and third devices.

17. The method of claim 15, wherein the information useful for generating a second 3D depth map includes the device identifying information, and the first and second 3D depth maps are generated by a single camera located in respective first location and second locations.

18. The method of claim 15, wherein the information useful for generating a second 3D depth map includes an image of the first device taken by the second device.

19. The method of claim 15, comprising:

receiving from at least one of the second and third devices information useful for generating a third 3D depth map including an image of the first device; and

generating an aggregate 3D depth map using the first, second, and third 3D depth maps.

20. The method of claim 15, comprising:

altering aspects of images in the first and second 3D depth maps for a reference location to render altered images in the aggregate 3D depth map.