System and method for depth map

• US 10,523,918 B2
• Filed: 12/04/2017
• Issued: 12/31/2019
• Est. Priority Date: 03/24/2017
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Original Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Inventors: Peri, Christopher A.
• Primary Examiner(s): Senfi, Behrooz M
• Application Number:
  US15/830,832
• Publication Number:
  US 20180278918A1
• Time in Patent Office: 757 Days
• Field of Search: 348 38
• CPC Class Codes: G06T 2207/20228 Disparity calculation for image-based rendering
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A method, electronic device, and non-transitory computer readable medium for transmitting information is provided. The method includes receiving, from each of two 360-degree cameras, image data. The method also includes synchronizing the received image data from each of the two cameras. Additionally, the method includes creating a depth map from the received the image data based in part on a distance between the two cameras. The method also includes generating multi-dimensional content by combining the created depth map with the synchronized image data of at least one of the two cameras.

Claims

1. A method comprising:

receiving, from each of two 360-degree cameras, image data;

synchronizing the image data from each of the two 360-degree cameras based on a time when the image data from each of the two 360-degree cameras is captured;

creating a depth map from the synchronized image data based in part on a distance between the two 360-degree cameras; and

generating multi-dimensional content by combining the created depth map to the synchronized image data of at least one of the two 360-degree cameras.

2. The method of claim 1, further comprising:

matching pixels, between the synchronized image data from each of the 360-degree cameras;

deriving a comparative distance between each of the matched pixels; and

generating the depth map based on the comparative distance.

3. The method of claim 1, wherein generating the multi-dimensional content comprises simulating three dimensional content of a 360-degree scene based on a perspective of one of the 360-degree cameras.

4. The method of claim 1, wherein the two 360-degree cameras are positioned in a vertical alignment with a first 360 degree camera above a second 360-degree camera.

5. The method of claim 1, wherein when the two 360-degree cameras are positioned adjacent each other, the method further comprising:

in response to deriving an object is occluded by one of the 360-degree cameras, determining a size of the object based on a second distance that the object is from a second object within a stereo view of the two 360-degree cameras.

6. The method of claim 1, wherein when the two 360-degree cameras are positioned adjacent each other, the method further comprising:

in response to deriving an object is in between the two 360-degree cameras, determining a size of the object based on a second distance that the object is from each of the 360-degree cameras.

7. The method of claim 1, further comprising,

determining a size of an objects by identifying the object based on object recognition.

8. An electronic device comprising:

a communication interface,

at least one processor coupled to the communication interface, the at least one processor configured to;

receive from each of two 360-degree cameras, image data;

synchronize the image data from each of the two 360-degree cameras based on a time when the image data from each of the two 360-degree cameras is captured;

create a depth map from the synchronized image data based in part on a distance between the two 360-degree cameras; and

generate multi-dimensional content by combining the created depth map with the synchronized image data of at least one of the two 360-degree cameras.

9. The electronic device of claim 8, wherein the at least one processor is further configured to:

match pixels, between the synchronized image data from each of the 360-degree cameras;

derive a comparative distance between each of the matched pixels; and

generate the depth map based on the comparative distance.

10. The electronic device of claim 8, wherein to generate the multi-dimensional content, the at least one processor is further configured to:

simulate three dimensional content of a 360-degree scene based on a perspective of one of the 360-degree cameras.

11. The electronic device of claim 8, wherein the two 360-degree cameras are positioned in a vertically alignment with a first 360-degree camera above a second 360-degree camera.

12. The electronic device of claim 8, wherein:

the two 360-degree cameras are positioned adjacent each other, and

the at least one processor is further configured to determine a size of an object based on a second distance the object is from a second object within a stereo view of the two 360-degree cameras, in response to deriving the object is occluded by one of the 360-degree cameras.

13. The electronic device of claim 8, wherein:

the two 360-degree cameras are positioned adjacent each other, and

the at least one processor is further configured to determine a size of an object based on a second distance the object is from each of the 360-degree cameras, in response to deriving the object is in between the two 360-degree cameras.

14. The electronic device of claim 8, wherein the at least one processor is further configured to determine a size of an objects by identifying the object based on object recognition.

15. A non-transitory computer readable medium comprising program code that, when executed by at least one processor, causes the at least one processor to:

receive from each of two 360-degree cameras, image data;

synchronize the image data from each of the two 360-degree cameras based on a time when the image data from each of the two 360-degree cameras is captured;

create a depth map from the synchronized image data based in part on a distance between the two 360-degree cameras; and

generate multi-dimensional content by combining the created depth map with the synchronized image data of at least one of the 360-degree two cameras.

16. The non-transitory computer readable medium of claim 15, further comprising program code that, when executed by the at least one processor, causes the at least one processor to:

match pixels, between the synchronized image data from each of the 360-degree cameras;

derive a comparative distance between each of the matched pixels; and

generate the depth map based on the comparative distance.

17. The non-transitory computer readable medium of claim 15, wherein to generate the multi-dimensional content, further comprises program code that, when executed by the at least one processor, causes the at least one processor to simulate three dimensional content of a 360-degree scene based on a perspective of one of the 360-degree cameras.

18. The non-transitory computer readable medium of claim 15, wherein the two 360-degree cameras are positioned adjacent each other, and further comprising program code that, when executed by the at least one processor, causes the at least one processor to determine a size of an object based on a second distance the object is from a second object within a stereo view of the two 360-degree cameras, in response to deriving the object is occluded by one of the 360-degree cameras.

19. The non-transitory computer readable medium of claim 15, wherein the two 360-degree cameras are positioned adjacent each other, and further comprising program code that, when executed by the at least one processor, causes the at least one processor to determine a size of an object based on a second distance the object is from each of the 360-degree cameras, in response to deriving the object is in between the two 360-degree cameras.

20. The non-transitory computer readable medium of claim 15, further comprising program code that, when executed by the at least one processor, causes the at least one processor to determine a size of an objects by identifying the object based on object recognition.