SYSTEM AND METHOD FOR CREATING 3D VIDEO

• US 20110074926A1
• Filed: 05/05/2010
• Published: 03/31/2011
• Est. Priority Date: 09/28/2009
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Bhat, Kong Posh, Khan, Farooq, Li, Ying
• Granted Patent: US 9,083,956 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: H04N 13/178 Metadata, e.g. disparity information
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion

Abstract

A system and method for generating 3D video from a plurality of 2D video streams is provided. A video capture device for capturing video to be transformed into 3D video includes a camera module for capturing a two-dimensional (2D) video stream, a location module for determining a location of the video capture device, an orientation module for determining an orientation of the video capture device, and a processing module for associating additional information with the 2D video stream captured by the camera module, the additional information including the orientation of the video capture device and the location of the video capture device.

Claims

1. A video capture device for capturing video to be transformed into 3D video, the video capture device comprising:

a camera module for capturing a two-dimensional (2D) video stream;

a location module for determining a location of the video capture device;

an orientation module for determining an orientation of the video capture device; and

a processing module for associating additional information with the 2D video stream captured by the camera module, the additional information including the orientation of the video capture device and the location of the video capture device.

2. The video capture device of claim 1, further comprising:

a rangefinder module for determining a distance between the video capture device and an object being photographed;

wherein the additional information includes the distance between the video capture device and the object being photographed.

3. The video capture device of claim 1, wherein the orientation module includes at least one of a gyroscope, a compass, and an accelerometer.

4. The video capture device of claim 1, wherein the location module comprises a Global Positioning System (GPS) receiver.

5. The video capture device of claim 1, further comprising:

a communication module for receiving at least one additional 2D video stream having appended additional information from at least one other video capture device; and

a three-dimensional (3D) video generation module for generating a 3D video based on the 2D video stream and the at least one additional 2D video stream.

6. The video capture device of claim 1, further comprising:

a communication module for transmitting the 2D video stream to a 3D video generation module and for receiving 3D video based on the 2D video stream from the 3D video generation module.

7. The video capture device of claim 1, wherein the processing module determines a distance between the video capture device and the object being filmed based on zoom and focus information of the camera module.

8. A system for generating 3D video based on a plurality of 2D video streams, the system comprising:

a plurality of video capture devices for capturing 2D video; and

a 3D video generator including a 3D video generation module for generating the 3D video based on the 2D video streams received from each of the plurality of video capture devices.

9. A three-dimensional (3D) video generator for generating 3D video from a plurality of two-dimensional (2D) video streams, the 3D video generator comprising:

a communication module for receiving a plurality of 2D video streams, each of the plurality of 2D video streams received from a different video capture device; and

a 3D video generation module for generating the 3D video based on the plurality of 2D video streams and additional information included in each of the plurality of 2D video streams, the additional information relating to the corresponding video capture device.

10. The 3D video generator of claim 9,

wherein the 3D video generation module synchronizes the plurality of 2D video streams using timing information in the additional information,

wherein the 3D video generation module divides the plurality of 2D video streams into groups, each group corresponding to a particular object or scene, and

wherein the 3D video generation module generates the 3D video corresponding to the particular object or scene based on the additional information.

11. The 3D video generator of claim 10, wherein the 3D video generation module divides the plurality of 2D video streams into groups based on at least one of stream ID information included in the additional information, image or pattern recognition of objects or scenes included in the plurality of 2D video streams, and coordinates of a scene being filmed.

12. The 3D video generator of claim 10, wherein the additional information comprises at least one of stream information of the corresponding 2D video stream, Global Positioning System (GPS) coordinates of the corresponding video capture device, orientation of the corresponding video capture device, timing information of the corresponding 2D video stream, and distance to an object or scene being filmed.

13. The 3D video generator of claim 10,

wherein the 3D video generation module identifies time segments in the plurality of 2D video streams in a group, the time segments being time segments in which two or more of the plurality of video capture devices captured video of the particular object or scene corresponding to the group,

wherein the 3D video generation module generates 3D video for the time segments in which two or more of the plurality of video devices captured video of the particular object or scene corresponding to the group, and

wherein the 3D video generation module generates 2D video for remaining time segments in which only one video capture device captured video of the particular object or scene corresponding to the group.

14. The 3D video generator of claim 10, wherein the 3D generation module determines a viewing angle for each of the plurality of video capture devices according to

15. The 3D video generator of claim 14, wherein the 3D video generation module determines a distance (a+e) between the first of the plurality of video capture devices and a third of the plurality of video capture devices, determines a distance (b+d) between the second of the plurality of video capture devices and a fourth of the plurality of video capture devices, determines a location of the target object based on the distance (a+e) and the distance (b+d);

and determines the distance a and the distance b based on the location of the target object.

16. The 3D video generator of claim 10, wherein the 3D video generation module determines viewing angles of each of the plurality of video capture devices based on the orientation information of the plurality of video capture devices, and generates the 3D video based on the additional information and the viewing angles.

17. A method of generating three-dimensional (3D) video, the method comprising:

capturing two-dimensional (2D) video frames;

generating a 2D video stream including the captured 2D video frames;

appending additional information about the 2D video stream to the 2D video stream;

transmitting the 2D video stream including the additional information to a 3D video generator; and

receiving a 3D video based on the 2D video stream from the 3D video generator.

18. The method of claim 17, wherein the additional information includes at least one of:

orientation information of a video capture device capturing the 2D video;

time information indicating a time when the 2D video is captured;

location information of the video capture device;

location information of an object being filmed;

a name of the object being filmed; and

a video stream ID of the 2D video stream.

19. The method of claim 18, wherein the location information is GPS coordinates.

20. The method of claim 17, further comprising:

storing the 2D video stream with the additional information prior to transmitting the 2D video stream with the additional information.

21. The method of claim 17, further comprising:

determining a distance between a video capture device capturing the 2D video and an object being filmed;

wherein the appending of the additional information comprises appending the distance between the video capture device and the object being filmed to the 2D video stream.

22. The method of claim 21, wherein the determining of the distance between the video capture device and the object being filmed comprises:

determining the distance between the video capture device and the object being filmed based on zoom and focus information of a camera of the video capture device.

23. The method of claim 17, wherein the appending of the additional information comprises appending additional information about each of the 2D video frames in the 2D video stream to the corresponding 2D video frame in the 2D video stream.

24. A method of generating three-dimensional (3D) video, the method comprising:

receiving 2D video streams from a plurality of video capture devices;

extracting additional information from each of the plurality of 2D video streams; and

generating the 3D video based on the plurality of 2D video streams and the extracted additional information.

25. The method of claim 24, further comprising:

synchronizing the plurality of 2D video streams based on timing information included in the additional information; and

grouping the plurality of 2D video streams into at least one group, based on a particular object or scene included in the plurality of video streams;

wherein the generating of the 3D video includes generating 3D video corresponding to the particular object or scene, based on a group including 2D video streams of the particular object or scene and additional information contained in the 2D video streams included in the group.

26. The method of claim 25, wherein the dividing of the plurality of 2D video streams into at least one group comprises:

dividing the plurality of 2D video streams into groups based on at least one of stream ID information included in the additional information, image or pattern recognition of the particular object or scene included in the plurality of 2D video streams, and location information of the particular scene.

27. The method of claim 25, further comprising:

identifying time segments in the plurality of 2D video streams in a group, the time segments corresponding to time segments in which two or more of the plurality of video capture devices captured video of the particular object or scene corresponding to the group; and

generating 2D video for remaining time segments in which only one of the plurality of video capture devices captured video of the particular object or scene corresponding to the group;

wherein the generating of the 3D video comprises generating the 3D video for time segments in which two or more of the plurality of video capture devices captured video of the particular object or scene corresponding to the group.

28. The method of claim 25, wherein the generating of the 3D video comprises:

generating the 3D video based on the additional information and on viewing angles of the plurality of video capture devices.

29. The method of claim 28, further comprising:

determining a viewing angle of each of the plurality of video capture devices according to

30. The method of claim 29, wherein the determining of the viewing angle comprises:

determining a distance (a+e) between the first of the plurality of video capture devices and a third of the plurality of video capture devices;

determining a distance (b+d) between the second of the plurality of video capture devices and a fourth of the plurality of video capture devices;

determining a location of the object being filmed based on the distance (a+e) and the distance (b+d); and

determining the distance a and the distance b based on the location of the particular object.

31. The method of claim 28, further comprising:

determining the viewing angles of each of the plurality of video capture devices based on orientation information of the plurality of video capture devices.