AUDIO DATA PROCESSING METHOD AND DEVICE

• US 20180247629A1
• Filed: 05/01/2018
• Published: 08/30/2018
• Est. Priority Date: 11/03/2015
• Status: Active Grant

• Current Assignee: Guangzhou Kugou Computer Technology Co. Ltd.
• Original Assignee: Guangzhou Kugou Computer Technology Co. Ltd.
• Inventors: ZHAO, Weifeng, CHEN, Xueqi
• Granted Patent: US 10,665,218 B2
• Time in Patent Office: Days
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 1/0091 Means for obtaining special acoustic effects combined with modulation G10H1/043
• CPC Class Codes: G10H 1/10 for obtaining chorus, celeste or ensemble effects continuous modulation G10H1/043
• CPC Class Codes: G10H 1/361 Recording/reproducing of accompaniment for use with an external source, e.g. karaoke systems
• CPC Class Codes: G10H 1/366 with means for modifying or correcting the external signal, e.g. pitch correction, reverberation, changing a singer's voice
• CPC Class Codes: G10H 2210/005 Musical accompaniment, i.e. complete instrumental rhythm synthesis added to a performed melody, e.g. as output by drum machines background music G10H2210/021
• CPC Class Codes: G10H 2210/061 for extraction of musical phrases, isolation of musically relevant segments, e.g. musical thumbnail generation, or for temporal structure analysis of a musical piece, e.g. determination of the movement sequence of a musical work
• CPC Class Codes: G10H 2210/066 for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
• CPC Class Codes: G10H 2210/251 Chorus, i.e. automatic generation of two or more extra voices added to the melody, e.g. by a chorus effect processor or multiple voice harmonizer, to produce a chorus or unison effect, wherein individual sounds from multiple sources with roughly the same timbre converge and are perceived as one
• CPC Class Codes: G10H 2210/331 Note pitch correction, i.e. modifying a note pitch or replacing it by the closest one in a given scale
• CPC Class Codes: G10H 2220/011 Lyrics displays, e.g. for karaoke applications
• CPC Class Codes: G10H 2240/325 Synchronizing two or more audio tracks or files according to musical features or musical timings synchronised lyrics, e.g. for karaoke G10H2220/011
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 21/003 Changing voice quality, e.g. pitch or formants
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal

Abstract

The present disclosure provides an audio data processing method performed at an electronic apparatus, the method including: obtaining a corresponding lyric file according to audio data to be processed; dividing the audio data according to a sentence in the lyric file to obtain an audio data segment; extracting data corresponding to an end syllable in the audio data segment; and performing harmonic processing on the data corresponding to the end syllable. In addition, further provided is an audio data processing device matching the method. The audio data processing method and device can prevent entire audio data from being attached with a harmonic sound effect during an entire time period, thereby improving authenticity of harmonic simulation.

Claims

1. An audio data processing method performed at an electronic apparatus having one or more processors and memory storing a plurality of program instructions be executed by the one or more processors, the method comprising:

obtaining a corresponding lyric file according to audio data to be processed;

dividing the audio data according to a sentence in the lyric file to obtain an audio data segment;

extracting data corresponding to an end syllable in the audio data segment; and

performing harmonic processing on the data corresponding to the end syllable.

2. The method according to claim 1, wherein the step of obtaining a corresponding lyric file according to audio data to be processed comprises:

obtaining a song according to the audio data to be processed; and

obtaining a lyric file corresponding to the song.

3. The method according to claim 1, wherein the step of dividing the audio data according to a sentence in the lyric file to obtain an audio data segment comprises:

extracting start time and end time of a sentence in the lyric file; and

dividing the audio data according to the start time and the end time to obtain an audio data segment corresponding to the sentence.

4. The method according to claim 3, wherein the step of extracting data corresponding to an end syllable in the audio data segment comprises:

extracting data corresponding to the end syllable in the audio data segment according to a preset end syllable length value.

5. The method according to claim 4, wherein the step of extracting data corresponding to the end syllable in the audio data segment according to a preset end syllable length value comprises:

obtaining end syllable start time according to end time corresponding to the sentence of the audio data segment and the preset end syllable length value;

locating start data corresponding to the end syllable in the audio data segment according to the end syllable start time and a sampling rate of the audio data; and

extracting data using the start data in the audio data segment as a start until an end of the audio data segment is extracted, to obtain data corresponding to the end syllable.

6. The method according to claim 1, wherein the step of performing harmonic processing on the data corresponding to the end syllable comprises:

using data corresponding to the end syllable as an original sound, copying the original sound, and performing falling tone processing to obtain intermediate data;

copying the intermediate data and obtaining multiple pieces of the intermediate data;

performing random delay and attenuation processing on each piece of intermediate data to obtain multiple pieces of harmonic sound effect data; and

mixing and superposing the original sound and the harmonic sound effect data to obtain simulative harmonic audio data.

7. An electronic apparatus having one or more processors, memory coupled to the one or more processors, and a plurality of program instructions stored in the memory that, when executed by the one or more processors, cause the electronic apparatus to perform the following operations:

obtaining a corresponding lyric file according to audio data to be processed;

dividing the audio data according to a sentence in the lyric file to obtain an audio data segment;

extracting data corresponding to an end syllable in the audio data segment; and

performing harmonic processing on the data corresponding to the end syllable.

8. The electronic apparatus according to claim 7, wherein the operation of obtaining a corresponding lyric file according to audio data to be processed comprises:

obtaining a song according to the audio data to be processed; and

obtaining a lyric file corresponding to the song.

9. The electronic apparatus according to claim 7, wherein the operation of dividing the audio data according to a sentence in the lyric file to obtain an audio data segment comprises:

extracting start time and end time of a sentence in the lyric file; and

dividing the audio data according to the start time and the end time to obtain an audio data segment corresponding to the sentence.

10. The electronic apparatus according to claim 9, wherein the operation extracting data corresponding to an end syllable in the audio data segment comprises:

extracting data corresponding to the end syllable in the audio data segment according to a preset end syllable length value.

11. The electronic apparatus according to claim 10, wherein the operation of extracting data corresponding to the end syllable in the audio data segment according to a preset end syllable length value comprises:

obtaining end syllable start time according to end time corresponding to the sentence of the audio data segment and the preset end syllable length value;

locating start data corresponding to the end syllable in the audio data segment according to the end syllable start time and a sampling rate of the audio data; and

extracting data using the start data in the audio data segment as a start until an end of the audio data segment is extracted, to obtain data corresponding to the end syllable.

12. The electronic apparatus according to claim 7, wherein the operation of performing harmonic processing on the data corresponding to the end syllable comprises:

using data corresponding to the end syllable as an original sound, copying the original sound, and performing falling tone processing to obtain intermediate data;

copying the intermediate data and obtaining multiple pieces of the intermediate data;

performing random delay and attenuation processing on each piece of intermediate data to obtain multiple pieces of harmonic sound effect data; and

mixing and superposing the original sound and the harmonic sound effect data to obtain simulative harmonic audio data.

13. A non-transitory computer readable storage medium storing a plurality of program instructions in connection with an electronic apparatus having one or more processors, wherein the plurality of program instructions, when executed by the one or more processors, cause the electronic apparatus to perform the following operations:

obtaining a corresponding lyric file according to audio data to be processed;

dividing the audio data according to a sentence in the lyric file to obtain an audio data segment;

extracting data corresponding to an end syllable in the audio data segment; and

performing harmonic processing on the data corresponding to the end syllable.

14. The non-transitory computer readable storage medium according to claim 13, wherein the operation of obtaining a corresponding lyric file according to audio data to be processed comprises:

obtaining a song according to the audio data to be processed; and

obtaining a lyric file corresponding to the song.

15. The non-transitory computer readable storage medium according to claim 13, wherein the operation of dividing the audio data according to a sentence in the lyric file to obtain an audio data segment comprises:

extracting start time and end time of a sentence in the lyric file; and

dividing the audio data according to the start time and the end time to obtain an audio data segment corresponding to the sentence.

16. The non-transitory computer readable storage medium according to claim 15, wherein the operation extracting data corresponding to an end syllable in the audio data segment comprises:

extracting data corresponding to the end syllable in the audio data segment according to a preset end syllable length value.

17. The non-transitory computer readable storage medium according to claim 16, wherein the operation of extracting data corresponding to the end syllable in the audio data segment according to a preset end syllable length value comprises:

obtaining end syllable start time according to end time corresponding to the sentence of the audio data segment and the preset end syllable length value;

locating start data corresponding to the end syllable in the audio data segment according to the end syllable start time and a sampling rate of the audio data; and

extracting data using the start data in the audio data segment as a start until an end of the audio data segment is extracted, to obtain data corresponding to the end syllable.

18. The non-transitory computer readable storage medium according to claim 13, wherein the operation of performing harmonic processing on the data corresponding to the end syllable comprises:

using data corresponding to the end syllable as an original sound, copying the original sound, and performing falling tone processing to obtain intermediate data;

copying the intermediate data and obtaining multiple pieces of the intermediate data;

performing random delay and attenuation processing on each piece of intermediate data to obtain multiple pieces of harmonic sound effect data; and

mixing and superposing the original sound and the harmonic sound effect data to obtain simulative harmonic audio data.