Text synchronization with audio
First Claim
Patent Images
1. A computing device that is configured to synchronize lyrics with music, comprising:
- a processor;
a memory in electronic communication with the processor;
instructions stored in the memory, the instructions being executable by the processor to;
identify a marker for singing segments in the music where a person is singing using a machine learning model;
identify a marker for break segments in proximity to the singing segments where the person is not singing using the machine learning model;
identify lyric segments in lyrics associated with the music, the lyric segments being divided by lyric breaks;
synchronize one of the lyric breaks with a marker of one of the break segments; and
synchronize at least one of the lyric segments to a marker of one of the singing segments.
1 Assignment
0 Petitions
Accused Products
Abstract
A technology for synchronizing text with audio includes analyzing the audio to identify voice segments in the audio where a human voice is present and to identify non-voice segments in proximity to the voice segments. Segmented text associated with the audio, having text segments, may be identified and synchronized to the voice segments.
33 Citations
18 Claims
-
1. A computing device that is configured to synchronize lyrics with music, comprising:
-
a processor; a memory in electronic communication with the processor; instructions stored in the memory, the instructions being executable by the processor to; identify a marker for singing segments in the music where a person is singing using a machine learning model; identify a marker for break segments in proximity to the singing segments where the person is not singing using the machine learning model; identify lyric segments in lyrics associated with the music, the lyric segments being divided by lyric breaks; synchronize one of the lyric breaks with a marker of one of the break segments; and synchronize at least one of the lyric segments to a marker of one of the singing segments. - View Dependent Claims (2, 3, 4)
-
-
5. A computer-implemented method, comprising:
-
analyzing audio, using a processor, to extract features from the audio and identify voice segments in the audio where a human voice is present and to identify non-voice segments in proximity to the voice segments based on the extracted features; identifying segmented text associated with the audio, the segmented text having text segments; synchronizing the text segments to the voice segments using the processor; and soliciting group-sourced corrections to correct the synchronizing of the text segments to the voice segments. - View Dependent Claims (6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16)
-
-
17. A non-transitory computer-readable medium comprising computer-executable instructions which, when executed by a processor, implement a system, comprising:
-
an audio analysis module configured to analyze audio to identify a voice segment in the audio where a human voice is present; a text analysis module configured to identify segments in text associated with the audio and identify the voice segment as trained using other audio; a correlation module configured to determine a number of the segments of the text to associate with the voice segment; and a synchronization module to associate the number of the segments of the text with the voice segment. - View Dependent Claims (18)
-
Specification