Automatic generation of video from spherical content using audio/visual analysis
First Claim
1. A method for generating an output video from spherical video content, the method comprising:
- storing, by a video server, a first spherical video obtained from a first camera system comprising a first sequence of spherical video frames, each having a first spherical field of viewstoring, by a video server, a second spherical video obtained from a second camera system comprising a second sequence of spherical video frames, each having a second spherical field of view;
processing, by the video server, the first spherical video to identify a target audio or visual feature of interest meeting one or more audio or visual criteria;
determining, by the video server, a first range of frames of the first spherical video having the target feature of interest;
determining, by the video server, a second range of frames of the second spherical video having the target feature of interest;
determining, by the video server, a first sequence of sub-frames from each of the first range of frames, each of the first sequence of sub-frames having a non-spherical field of view, and each of the first sequence of sub-frames including a spatial region around the target feature of interest;
determining, by the video server, a second sequence of sub-frames from each of the second range of frames, each of the second sequence of sub-frames having a non-spherical field of view, and each of the second sequence of sub-frames including a spatial region around the target feature of interest;
generating, by the video server, a first combined sequence of sub-frames including the target feature of interest, the combined sequence of sub-frames comprising the first sequence of sub-frames and the second sequence of sub-frames;
generating, by the video server, a first portion of an output video including the first combined sequence of sub-frames; and
outputting the output video.
3 Assignments
0 Petitions
Accused Products
Abstract
A spherical content capture system captures spherical video content. A spherical video sharing platform enables users to share the captured spherical content and enables users to access spherical content shared by other users. In one embodiment, captured metadata or video/audio processing is used to identify content relevant to a particular user based on time and location information. The platform can then generate an output video from one or more shared spherical content files relevant to the user. The output video may include a non-spherical reduced field of view such as those commonly associated with conventional camera systems. Particularly, relevant sub-frames having a reduced field of view may be extracted from each frame of spherical video to generate an output video that tracks a particular individual or object of interest.
43 Citations
19 Claims
-
1. A method for generating an output video from spherical video content, the method comprising:
-
storing, by a video server, a first spherical video obtained from a first camera system comprising a first sequence of spherical video frames, each having a first spherical field of view storing, by a video server, a second spherical video obtained from a second camera system comprising a second sequence of spherical video frames, each having a second spherical field of view; processing, by the video server, the first spherical video to identify a target audio or visual feature of interest meeting one or more audio or visual criteria; determining, by the video server, a first range of frames of the first spherical video having the target feature of interest; determining, by the video server, a second range of frames of the second spherical video having the target feature of interest; determining, by the video server, a first sequence of sub-frames from each of the first range of frames, each of the first sequence of sub-frames having a non-spherical field of view, and each of the first sequence of sub-frames including a spatial region around the target feature of interest; determining, by the video server, a second sequence of sub-frames from each of the second range of frames, each of the second sequence of sub-frames having a non-spherical field of view, and each of the second sequence of sub-frames including a spatial region around the target feature of interest; generating, by the video server, a first combined sequence of sub-frames including the target feature of interest, the combined sequence of sub-frames comprising the first sequence of sub-frames and the second sequence of sub-frames; generating, by the video server, a first portion of an output video including the first combined sequence of sub-frames; and outputting the output video. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A non-transitory computer-readable storage medium storing instructions for generating an output video from spherical video content, the instructions when executed by one or more processors causing the one or more processors to perform steps including:
-
storing a first spherical video obtained from a first camera system comprising a first sequence of spherical video frames each having a first spherical field of view; storing a second spherical video obtained from a second camera system comprising a second sequence of spherical video frames, each having a second spherical field of view; processing the first spherical video to identify a target audio or visual feature of interest meeting one or more audio or visual criteria; determining a first range of frames of the first spherical video having the target feature of interest; determining a second range of frames of the second spherical video having the target feature of interest; determining a first sequence of sub-frames from each of the first range of frames, each of the first sequence of sub-frames having a non-spherical field of view, and each of the first sequence of sub-frames including a spatial region around the target feature of interest; determining a second sequence of sub-frames from each of the second range of frames, each of the second sequence of sub-frames having a non-spherical field of view, and each of the second sequence of sub-frames including a spatial region around the target feature of interest; generating a first combined sequence of sub-frames including the target feature of interest comprising of the first sequence of sub-frames and the second sequence of sub-frames; generating a first portion of an output video including the first combined sequence of sub-frames; and outputting the output video. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. A video server for generating an output video from spherical video content, the video server comprising:
-
one or more processors; and a non-transitory computer-readable storage medium storing instructions that when executed by the one or more processors causing the one or more processors to perform steps including; storing a first spherical video from a first camera system comprising a first sequence of spherical video frames each having a first spherical field of view; storing a second spherical video from a second camera system comprising a second sequence of spherical video frames, each having a second spherical field of view; processing the first spherical video to identify a target audio or visual feature of interest meeting one or more audio or visual criteria; determining a first range of frames of the first spherical video having the target feature of interest; determining, by the video server, a second range of frames of the second spherical video having the target feature of interest; determining a first sequence of sub-frames from each of the first range of frames, each of the first sequence of sub-frames having a non-spherical field of view, and each of the first sequence of sub-frames including a spatial region around the target feature of interest; determining a second sequence of sub-frames from each of the second range of frames, each of the second sequence of sub-frames having a non-spherical field of view, and each of the second sequence of sub-frames including a spatial region around the target feature of interest; generating a first combined sequence of sub-frames including the target feature of interest comprising of the first sequence of sub-frames and the second sequence of sub-frames; generating a first portion of an output video including the first combined sequence of sub-frames; and outputting the output video. - View Dependent Claims (16, 17, 18, 19)
-
Specification