Automatic generation of video and directional audio from spherical content
First Claim
1. A method for generating a video with corresponding audio, the method performing by a computing system including one or more processors, the method comprising:
- receiving, by the computing system, frames of a video having a field of view, the frames including a target;
receiving, by the computing system, audio signals captured concurrently with the frames;
determining, by the computing system, a time-varying path of the target within the frames based on an analysis of content of the frames or information associated with the video;
identifying, by the computing system, sub-frames within the frames based on the time-varying path of the target, the sub-frames including the target and having a reduced field of view relative to the field of view;
generating, by the computing system, an audio stream from the audio signals based on the time-varying path of the target, the audio stream including portions of one or more of the audio signals corresponding to a direction of the target; and
generating, by the computing system, an output video including the sub-frames and the audio stream.
3 Assignments
0 Petitions
Accused Products
Abstract
A spherical content capture system captures spherical video and audio content. In one embodiment, captured metadata or video/audio processing is used to identify content relevant to a particular user based on time and location information. The platform can then generate an output video from one or more shared spherical content files relevant to the user. The output video may include a non-spherical reduced field of view such as those commonly associated with conventional camera systems. Particularly, relevant sub-frames having a reduced field of view may be extracted from each frame of spherical video to generate an output video that tracks a particular individual or object of interest. For each sub-frame, a corresponding portion of an audio track is generated that includes a directional audio signal having a directionality based on the selected sub-frame.
-
Citations
20 Claims
-
1. A method for generating a video with corresponding audio, the method performing by a computing system including one or more processors, the method comprising:
-
receiving, by the computing system, frames of a video having a field of view, the frames including a target; receiving, by the computing system, audio signals captured concurrently with the frames; determining, by the computing system, a time-varying path of the target within the frames based on an analysis of content of the frames or information associated with the video; identifying, by the computing system, sub-frames within the frames based on the time-varying path of the target, the sub-frames including the target and having a reduced field of view relative to the field of view; generating, by the computing system, an audio stream from the audio signals based on the time-varying path of the target, the audio stream including portions of one or more of the audio signals corresponding to a direction of the target; and generating, by the computing system, an output video including the sub-frames and the audio stream. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A non-transitory computer-readable storage medium storing instructions for generating a video with corresponding audio, the instructions when executed by one or more processors causing the one or more processors to perform steps including:
-
receiving frames of a video having a field of view, the frames including a target; receiving audio signals captured concurrently with the frames; determining a time-varying path of the target within the frames based on an analysis of content of the frames or information associated with the video; identifying sub-frames within the frames based on the time-varying path of the target, the sub-frames including the target and having a reduced field of view relative to the field of view; generating an audio stream from the audio signals based on the time-varying path of the target, the audio stream including portions of one or more of the audio signals corresponding to a direction of the target; and generating an output video including the sub-frames and the audio stream. - View Dependent Claims (10, 11, 12, 13, 14)
-
-
15. A system for generating a video with corresponding audio, the system comprising:
- one or more processors; and
a non-transitory computer-readable storage medium storing instructions that when executed by the one or more processors causes the one or more processors to perform steps including; receiving frames of a video having a field of view, the frames including a target;
receiving audio signals captured concurrently with the frames;determining a time-varying path of the target within the frames based on an analysis of content of the frames or information associated with the video; identifying sub-frames within the frames based on the time-varying path of the target, the sub-frames including the target and having a reduced field of view relative to the field of view; generating an audio stream from the audio signals based on the time-varying path of the target, the audio stream including portions of one or more of the audio signals corresponding to a direction of the target; and generating an output video including the sub-frames and the audio stream. - View Dependent Claims (16, 17, 18, 19, 20)
- one or more processors; and
Specification