System and method for creating 3D video

• US 9,083,956 B2
• Filed: 05/05/2010
• Issued: 07/14/2015
• Est. Priority Date: 09/28/2009
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Khan, Farooq, Bhat, Kong Posh, Li, Ying
• Primary Examiner(s): Chio, Tat Chi
• Assistant Examiner(s): TRUONG, NGUYEN T
• Application Number:
  US12/774,295
• Publication Number:
  US 20110074926A1
• Time in Patent Office: 1,896 Days
• Field of Search: 348 46- 48, 348/55, 396/325, 382/154
• US Class Current: 1/1
• CPC Class Codes: H04N 13/178 Metadata, e.g. disparity information
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion

Abstract

A system and method for generating 3D video from a plurality of 2D video streams is provided. A video capture device for capturing video to be transformed into 3D video includes a camera module for capturing a two-dimensional (2D) video stream, a location module for determining a location of the video capture device, an orientation module for determining an orientation of the video capture device, and a processing module for associating additional information with the 2D video stream captured by the camera module, the additional information including the orientation of the video capture device and the location of the video capture device.

Claims

1. A video capture device for capturing video to be transformed into 3D video, the video capture device comprising:

a camera module for capturing a two-dimensional (2D) video stream;

a location module for determining a location of the video capture device;

an orientation module for determining an orientation of the video capture device;

a processing module for associating additional information with each frame of the 2D video stream captured by the camera module, the additional information including the orientation of the video capture device and the location of the video capture device,

a communication module for receiving at least one additional 2D video stream having appended additional information for each frame of the at least one additional 2D video stream from at least one other video capture device that is not under the control of the video capture device; and

a three-dimensional (3D) video generation module for generating a 3D video based on the 2D video stream and the at least one additional 2D video stream,

wherein the 3D video is generated based on a viewing angle for each of a plurality of video capture devices that is calculated according to

2. The video capture device of claim 1, further comprising:

a rangefinder module for determining a distance between the video capture device and an object being photographed;

wherein the additional information includes the distance between the video capture device and the object being photographed.

3. The video capture device of claim 1, wherein the orientation module includes at least one of a gyroscope, a compass, and an accelerometer.

4. The video capture device of claim 1, wherein the location module comprises a Global Positioning System (GPS) receiver.

5. The video capture device of claim 1, further comprising:

a communication module for transmitting the 2D video stream to a 3D video generation module and for receiving 3D video based on the 2D video stream from the 3D video generation module.

6. The video capture device of claim 1, wherein the processing module determines a distance between the video capture device and the object being filmed based on zoom and focus information of the camera module.

7. A system for generating 3D video based on a plurality of 2D video streams, the system comprising:

a plurality of video capture devices for capturing 2D video and associating additional information with each frame of the 2D video; and

a 3D video generator including a 3D video generation module for generating the 3D video based on the 2D video streams, a viewing angle for each of the plurality of video capture devices, and the additional information received from each of the plurality of video capture devices,

wherein none of the video capture devices are controlled by any other of the video capture devices, and

wherein the 3D generation module determines the viewing angle for each of the plurality of video capture devices according to

8. A three-dimensional (3D) video generator for generating 3D video from a plurality of two-dimensional (2D) video streams, the 3D video generator comprising:

a communication module for receiving a plurality of 2D video streams, each of the plurality of 2D video streams received from a different video capture device that is not under the control over the 3D video generator; and

a 3D video generation module for generating the 3D video based on the plurality of 2D video streams, a viewing angle for each of a plurality of video capture devices, and additional information for each frame of the plurality of 2D video streams included in each of the plurality of 2D video streams, the additional information relating to the corresponding video capture device,

wherein the 3D generation module determines the viewing angle for each of the plurality of video capture devices according to

9. The 3D video generator of claim 8,

wherein the 3D video generation module synchronizes the plurality of 2D video streams using timing information in the additional information,

wherein the 3D video generation module divides the plurality of 2D video streams into groups, each group corresponding to the particular object or a scene, and

wherein the 3D video generation module generates the 3D video corresponding to the particular object or scene based on the additional information.

10. The 3D video generator of claim 9, wherein the 3D video generation module divides the plurality of 2D video streams into groups based on at least one of stream ID information included in the additional information, image or pattern recognition of objects or scenes included in the plurality of 2D video streams, and coordinates of a scene being filmed.

11. The 3D video generator of claim 9, wherein the additional information comprises at least one of stream information of the corresponding 2D video stream, Global Positioning System (GPS) coordinates of the corresponding video capture device, orientation of the corresponding video capture device, timing information of the corresponding 2D video stream, and distance to an object or scene being filmed.

12. The 3D video generator of claim 9,

wherein the 3D video generation module identifies time segments in the plurality of 2D video streams in a group, the time segments being time segments in which two or more of the plurality of video capture devices captured video of the particular object or scene corresponding to the group,

wherein the 3D video generation module generates 3D video for the time segments in which two or more of the plurality of video devices captured video of the particular object or scene corresponding to the group, and

wherein the 3D video generation module generates 2D video for remaining time segments in which only one video capture device captured video of the particular object or scene corresponding to the group.

13. The 3D video generator of claim 9, wherein the 3D video generation module determines a distance (a +e) between the first of the plurality of video capture devices and a third of the plurality of video capture devices, determines a distance (b +d) between the second of the plurality of video capture devices and a fourth of the plurality of video capture devices, determines a location of the target object based on the distance (a +e) and the distance (b +d);

and determines the distance a and the distance b based on the location of the target object.

14. The 3D video generator of claim 9, wherein the 3D video generation module determines the viewing angles of each of the plurality of video capture devices based on the orientation information of the plurality of video capture devices.

15. A method of generating three-dimensional (3D) video, the method comprising:

capturing two-dimensional (2D) video frames by a video capture device;

generating a 2D video stream including the captured 2D video frames;

appending additional information about the 2D video stream to each frame of the 2D video stream;

transmitting the 2D video stream including the additional information to a 3D video generator that is not under the control of, or controlled by, the video capture device; and

receiving a 3D video based on the 2D video stream from the 3D video generator,

wherein the 3D video includes a plurality of time segments, the time segments in which two or more video devices captured video including 3D video, and the time segments in which only one video capture device captured video including 2D video, and

wherein the 3D video is generated according to a viewing angle of each of a plurality of video capture devices that is calculated according to

16. The method of claim 15, wherein the additional information includes at least one of:

orientation information of a video capture device capturing the 2D video;

time information indicating a time when the 2D video is captured;

location information of the video capture device;

location information of an object being filmed;

a name of the object being filmed; and

a video stream ID of the 2D video stream.

17. The method of claim 16, wherein the location information is GPS coordinates.

18. The method of claim 15, further comprising:

storing the 2D video stream with the additional information prior to transmitting the 2D video stream with the additional information.

19. The method of claim 15, further comprising:

determining a distance between a video capture device capturing the 2D video and an object being filmed;

wherein the appending of the additional information comprises appending the distance between the video capture device and the object being filmed to the 2D video stream.

20. The method of claim 19, wherein the determining of the distance between the video capture device and the object being filmed comprises:

determining the distance between the video capture device and the object being filmed based on zoom and focus information of a camera of the video capture device.

21. A method of generating three-dimensional (3D) video, the method comprising:

receiving, by a 3D video generator, 2D video streams from a plurality of video capture devices, no more than one of which is under the control of the 3D video generator;

extracting additional information from each frame of the plurality of 2D video streams;

determining a viewing angle of each of the plurality of video capture devices according to

22. The method of claim 21, further comprising:

synchronizing the plurality of 2D video streams based on timing information included in the additional information; and

grouping the plurality of 2D video streams into at least one group, based on the particular object or a scene included in the plurality of video streams;

wherein the generating of the 3D video includes generating 3D video corresponding to the particular object or scene, based on a group including 2D video streams of the particular object or scene and additional information contained in the 2D video streams included in the group.

23. The method of claim 22, wherein the dividing of the plurality of 2D video streams into at least one group comprises:

dividing the plurality of 2D video streams into groups based on at least one of stream ID information included in the additional information, image or pattern recognition of the particular object or scene included in the plurality of 2D video streams, and location information of the particular scene.

24. The method of claim 22, further comprising:

identifying time segments in the plurality of 2D video streams in a group, the time segments corresponding to time segments in which two or more of the plurality of video capture devices captured video of the particular object or scene corresponding to the group; and

generating 2D video for remaining time segments in which only one of the plurality of video capture devices captured video of the particular object or scene corresponding to the group;

wherein the generating of the 3D video comprises generating the 3D video for time segments in which two or more of the plurality of video capture devices captured video of the particular object or scene corresponding to the group.

25. The method of claim 21, wherein the determining of the viewing angle comprises:

determining a distance (a +e) between the first of the plurality of video capture devices and a third of the plurality of video capture devices;

determining a distance (b +d) between the second of the plurality of video capture devices and a fourth of the plurality of video capture devices;

determining a location of the object being filmed based on the distance (a +e) and the distance (b +d); and

determining the distance a and the distance b based on the location of the particular object.

26. The method of claim 22, further comprising:

determining the viewing angles of each of the plurality of video capture devices based on orientation information of the plurality of video capture devices.