Generating a 3D video of an event using crowd sourced data

• US 10,033,992 B1
• Filed: 09/09/2014
• Issued: 07/24/2018
• Est. Priority Date: 09/09/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: McGowan, John
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Dang, Philip
• Application Number:
  US14/481,120
• Time in Patent Office: 1,414 Days
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/361 Reproducing mixed stereoscopic images; Reproducing mixed monoscopic and stereoscopic images, e.g. a stereoscopic image overlay window on a monoscopic image background

Abstract

Generating 3D content is described. A method includes identifying a group of videos of an event. The method includes detecting a first reference point in a first video and a second video from the group of videos. The method also includes creating a first three dimensional video frame based on a first video frame of the first video and a second video frame of the second video using the first reference point. The method further includes creating a second three dimensional video frame based on a third video frame of the first video and a fourth video frame of the second video using a second reference point. The method includes creating a three dimensional video by combining the first three dimensional frame and the second three dimensional frame in a sequential order based on respective timestamps of the first three dimensional frame and the second three dimensional frame.

Claims

1. A method comprising:

identifying, from a plurality of videos and by a processing device, a group of videos of an event from a plurality of users, wherein each video of the group of videos comprises a plurality of video frames;

detecting a first reference point in a first video and a second video from the group of videos that are from the plurality of users, the first reference point corresponding to a similarity in audio content of the first video and the second video;

creating a first three dimensional video frame based on a first video frame of the first video and a second video frame of the second video using a respective viewpoint associated with the first reference point from each of the first video frame and the second video frame;

creating a second three dimensional video frame based on a third video frame of the first video and a fourth video frame of the second video using a second reference point;

creating a three dimensional video by combining the first three dimensional video frame and the second three dimensional video frame in a sequential order based on respective timestamps of the first three dimensional video frame and the second three dimensional video frame; and

providing a notification to a respective user of the plurality of users indicating that the first video or the second video from the group of videos that is from the respective user has been used to create the three dimensional video.

2. The method of claim 1, wherein identifying, from the plurality of videos, the group of videos of the event comprises:

identifying metadata associated with each video of the plurality of videos; and

determining that the group of videos is associated with the event based on a threshold number of videos having metadata associated with a time and location, and wherein the threshold number is defined by the time and location of the event.

3. The method of claim 2, wherein determining that the group of videos is associated with the event comprises:

identifying a first set of videos that were captured within a predetermined amount of time; and

identifying, from the first set of videos, a second set of videos that were captured within a close proximity relative to each other.

4. The method of claim 2 further comprising indexing the metadata associated with each video of the plurality of videos before determining that the group of videos is associated with the event, wherein determining that the group of videos is associated with the event comprises identifying, in the index, the threshold number of videos that have metadata associated with the time and location.

5. The method of claim 1 wherein identifying, from the plurality of videos, the group of videos of the event comprises:

identifying a time and location of the event; and

identifying the group of videos based on the time and location of the event.

6. The method of claim 1 further comprising segmenting the first video and the second video into video frames.

7. The method of claim 1 further comprising providing the three dimensional video via a content sharing platform.

8. The method of claim 1, wherein each video of the group of videos was captured by a user device at the event.

9. The method of claim 1, wherein the first reference point is the same as the second reference point.

19. The non-transitory machine-readable storage medium of claim 1, wherein the operations further comprise:

segmenting the first video and the second video into video frames.

10. An apparatus comprising:

a memory; and

a processing device, communicably coupled to the memory, the processing device to execute instructions to;

identify, from a plurality of videos, a group of videos of an event from a plurality of users, wherein each video of the group of videos comprises a plurality of video frames;

detect a first reference point in a first video and a second video from the group of videos that are from the plurality of users, the first reference point corresponding to a similarity in audio content of the first video and the second video;

create a first three dimensional video frame based on a first video frame of the first video and a second video frame of the second video using a respective viewpoint associated with the first reference point from each of the first video frame and the second video frame;

create a second three dimensional video frame based on a third video frame of the first video and a fourth video frame of the second video using a second reference point;

create a three dimensional video by combining the first three dimensional video frame and the second three dimensional video frame in a sequential order based on respective timestamps of the first three dimensional video frame and the second three dimensional video frame; and

provide a notification to a respective user of the plurality of users indicating that the first video or the second video from the group of videos that is from the respective user has been used to create the three dimensional video.

11. The apparatus of claim 10, wherein when identifying, from the plurality of videos, the group of videos of the event, the processing device is to:

identify metadata associated with each video of the plurality of videos; and

determine that the group of videos is associated with the event based on a threshold number of videos having metadata associated with a time and location, and wherein the threshold number is defined by the time and location of the event.

12. The apparatus of claim 11, wherein when determining that the group of videos is associated with the event, the processing device is to:

identify a first set of videos that were captured within a predetermined amount of time; and

identify, from the first set of videos, a second set of videos that were captured within a close proximity relative to each other.

13. The apparatus of claim 10, wherein when identifying, from the plurality of videos, the group of videos of the event, the processing device is to:

identify a time and location of the event; and

identify the group of videos based on the time and location of the event.

14. A non-transitory machine-readable storage medium storing instructions which, when executed, cause a processing device to perform operations comprising:

identifying, from a plurality of videos, a group of videos of an event from a plurality of users, wherein each video of the group of videos comprises a plurality of video frames;

detecting a first reference point in a first video and a second video from the group of videos that are from the plurality of users, the first reference point corresponding to a similarity in audio content of the first video and the second video;

creating a first three dimensional video frame based on a first video frame of the first video and a second video frame of the second video using a respective viewpoint associated with the first reference point from each of the first video frame and the second video frame;

creating a second three dimensional video frame based on a third video frame of the first video and a fourth video frame of the second video using a second reference point;

creating a three dimensional video by combining the first three dimensional video frame and the second three dimensional video frame in a sequential order based on respective timestamps of the first three dimensional video frame and the second three dimensional video frame; and

providing a notification to a respective user of the plurality of users indicating that the first video or the second video from the group of videos that is from the respective user has been used to create the three dimensional video.

15. The non-transitory machine-readable storage medium of claim 14, wherein to identify, from the plurality of videos, the group of videos of the event, the operations further comprise:

identifying metadata associated with each video of the plurality of videos; and

determining that the group of videos is associated with the event based on a threshold number of videos having metadata associated with a time and location, and wherein the threshold number is defined by the time and location of the event.

16. The non-transitory machine-readable storage medium of claim 15, wherein to determine that the group of videos is associated with the event, the operations further comprise:

identifying a first set of videos that were captured within a predetermined amount of time; and

identifying, from the first set of videos, a second set of videos that were captured within a close proximity relative to each other.

17. The non-transitory machine-readable storage medium of claim 15, wherein the operations further comprise:

indexing the metadata associated with each video of the plurality of videos before determining that the group of videos is associated with the event, wherein determining that the group of videos is associated with the event comprises identifying, in the index, the threshold number of videos that have metadata associated with the time and location.

18. The non-transitory machine-readable storage medium of claim 14, wherein the first reference point is the same as the second reference point.