Sound processing apparatus and method, and program therefor

• US 7,945,446 B2
• Filed: 03/09/2006
• Issued: 05/17/2011
• Est. Priority Date: 03/10/2005
• Status: Expired due to Fees

• Current Assignee: Yamaha Corporation
• Original Assignee: Yamaha Corporation
• Inventors: Yoshioka, Yasuo, Kemmochi, Hideki, Bonada, Jordi
• Primary Examiner(s): Dorvil; Richemond
• Assistant Examiner(s): ORTIZ SANCHEZ, MICHAEL
• Application Number:
  US11/372,812
• Publication Number:
  US 20060212298A1
• Time in Patent Office: 1,895 Days
• Field of Search: 704/278
• US Class Current: 704/278
• CPC Class Codes: G10H 1/10 for obtaining chorus, celeste or ensemble effects continuous modulation G10H1/043
• CPC Class Codes: G10H 1/366 with means for modifying or correcting the external signal, e.g. pitch correction, reverberation, changing a singer's voice
• CPC Class Codes: G10H 2210/251 Chorus, i.e. automatic generation of two or more extra voices added to the melody, e.g. by a chorus effect processor or multiple voice harmonizer, to produce a chorus or unison effect, wherein individual sounds from multiple sources with roughly the same timbre converge and are perceived as one
• CPC Class Codes: G10H 2250/031 Spectrum envelope processing
• CPC Class Codes: G10H 5/005 Voice controlled instruments
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser

Abstract

Spectrum envelope of an input sound is detected. In the meantime, a converting spectrum is acquired which is a frequency spectrum of a converting sound comprising a plurality of sounds, such as unison sounds. Output spectrum is generated by imparting the detected spectrum envelope of the input sound to the acquired converting spectrum. Sound signal is synthesized on the basis of the generated output spectrum. Further, a pitch of the input sound may be detected, and frequencies of peaks in the acquired converting spectrum may be varied in accordance with the detected pitch of the input sound. In this manner, the output spectrum can have the pitch and spectrum envelope of the input sound and spectrum frequency components of the converting sound comprising a plurality of sounds, and thus, unison sounds can be readily generated with simple arrangements.

Claims

1. A sound processing apparatus comprising:

a pitch detection section that detects a pitch of an input sound;

an envelope detection section that detects a spectrum envelope of the input sound;

a spectrum acquisition section that acquires converting spectrums that are frequency spectrums of a converting sound comprising a plurality of sounds;

a spectrum conversion section that generates output spectrums created by imparting the spectrum envelope of the input sound, detected by said envelope detection section, to the converting spectrums acquired by said spectrum acquisition section; and

a sound synthesis section that synthesizes a sound signal on the basis of the output spectrums generated by said spectrum conversion section,

wherein said plurality of sounds included in said converting sound have been sounded in parallel from different sound sources, said converting spectrums having a plurality of peaks, a band of each of the peaks being wider than a corresponding band of each peak of spectrums of the input sound, and

wherein, as data representative of the converting spectrums acquired by said spectrum acquisition section, said spectrum acquisition section supplies, for each spectrum composing the converting spectrums, unit data composed of data indicative of a frequency and intensity of the spectrum at the frequency, and

wherein said spectrum conversion section includes;

a pitch conversion section that varies, in accordance with the pitch of the input sound detected by said pitch detection section, the frequency indicated by the unit data of each spectrum of the converting spectrums; and

an envelope adjustment section that adjusts a spectrum envelope of the converting spectrums, having the frequencies varied by said pitch conversion section, to substantially agree with the spectrum envelope of the input sound detected by said envelope detection section,

wherein, for each of the peaks in the converting spectrums, said envelope adjustment section

determines an intensity of a frequency in the spectrum envelope of the input sound, said frequency in the spectrum envelope of the input sound corresponding to the frequency of the peak in the converting spectrum,

determines a ratio between the determined intensity of the frequency in the spectrum envelope of the input sound and the intensity of a frequency in the converting spectrums as indicated by the unit data, said frequency in the converting spectrum corresponding to the peak in the converting spectrum, and

multiplies an intensity of the unit data of each spectrum belonging to the band centered around said peak in the converting spectrums, by the determined ratio,

wherein the spectrum envelope of the converting spectrums substantially agree with the spectrum envelope of the input sound.

2. A sound processing apparatus as claimed in claim 1 wherein said pitch conversion section expands or reduces a whole of the converting spectrum in accordance with the pitch of the input sound detected by said pitch detection section.

3. A sound processing apparatus as claimed in claim 1 wherein said pitch conversion section displaces the frequency of each of the peaks in accordance with the pitch of the input sound while maintaining spectrum distribution regions formed around each of the peaks.

4. A sound processing apparatus as claimed in claim 1 wherein said spectrum acquisition section acquires converting spectrums of a converting sound, among a plurality of the converting sounds differing from each other in fundamental pitch, which has a fundamental pitch closest to the pitch detected by said pitch detection section.

5. A sound processing apparatus as claimed in claim 1 wherein the converting sound of the converting spectrums acquired by said spectrum acquisition section comprises a plurality of sounds uttered in unison.

6. A sound processing apparatus as claimed in claim 1 wherein said spectrum acquisition section acquires the converting spectrums that vary over time.

7. A sound processing apparatus as claimed in claim 1 wherein said sound synthesis section synthesizes a sound signal based on the output spectrums as long as generation of the input sound lasts.

8. A sound processing apparatus as claimed in claim 7 wherein said spectrum acquisition section sequentially acquires a limited plurality of the converting spectrums in accordance with passage of time, and said spectrum acquisition section re-acquires any of the limited plurality of the converting spectrums as long as the generation of the input sound lasts.

9. A sound processing apparatus as claimed in claim 1 which is provided as an attachment to a karaoke apparatus, and wherein the input sound is a sound signal picked up by a microphone of the karaoke apparatus.

10. A sound processing apparatus comprising:

a pitch detection section that detects a pitch of an input sound;

an envelope detection section that detects a spectrum envelope of the input sound;

a spectrum acquisition section that acquires converting spectrums that are frequency spectrums of a converting sound;

a spectrum conversion section that generates output spectrums created by imparting the spectrum envelope of the input sound, detected by said envelope detection section, to the converting spectrums acquired by said spectrum acquisition section; and

a sound synthesis section that synthesizes a sound signal on the basis of the output spectrum generated by said spectrum conversion section,

wherein said spectrum acquisition section selectively acquires either one of first converting spectrums that are frequency spectrums of a first converting sound having a plurality of peaks, or second converting spectrums that are frequency spectrums of a second converting sound having substantially a same pitch as the first converting sound indicated by said first converting spectrums and having a plurality of peaks of a greater bandwidth at each peak than said first converting spectrums, at least said second converting sound comprising a plurality of sounds which have been sounded in parallel from different sound sources, a band of each of the peaks in at least said second converting spectrums being wider than a band of each peak of spectrums of the input sound, and

wherein, as data representative of the converting spectrums acquired by said spectrum acquisition section, said spectrum acquisition section supplies, for each spectrum composing the converting spectrums, unit data composed of data indicative of a frequency and intensity of the spectrum at the frequency, and

wherein said spectrum conversion section includes;

a pitch conversion section that varies, in accordance with the pitch of the input sound detected by said pitch detection section, the frequency indicated by the unit data of each spectrum of the converting spectrums supplied by said spectrum acquisition section; and

an envelope adjustment section that adjusts a spectrum envelope of the converting spectrums, having the frequencies varied by said pitch conversion section, to substantially agree with the spectrum envelope of the input sound detected by said envelope detection section,

wherein, for each of the peaks in the converting spectrums, said envelope adjustment section

determines an intensity of a frequency in the spectrum envelope of the input sound, said frequency in the spectrum envelope of the input sound corresponding to the frequency of the peak in the converting spectrum,

determines a ratio between the determined intensity of the frequency in the spectrum envelope of the input sound and the intensity of a frequency in the converting spectrums as indicated by the unit data, said frequency in the converting spectrum corresponding to the peak in the converting spectrum, and

multiplies an intensity of the unit data of each spectrum belonging to the band centered around said peak in the converting spectrums, by the determined ratio,

wherein the spectrum envelope of the converting spectrums substantially agree with the spectrum envelope of the input sound.

11. A sound processing apparatus 10 wherein said first converting sound comprises a single sound.

12. A sound processing apparatus 10 wherein the first converting sound comprises a plurality of sounds different from the second converting sound.

13. A method for processing an input sound, said method comprising:

a step of detecting a pitch of an input sound;

a step of detecting a spectrum envelope of the input sound;

a step of acquiring converting spectrums that is frequency spectrums of a converting sound comprising a plurality of sounds;

a step of generating output spectrums created by imparting the spectrum envelope of the input sound, detected by said step of detecting, to the converting spectrums acquired by said step of acquiring; and

a step of synthesizing a sound signal on the basis of the output spectrums generated by said step of generating,

wherein said plurality of sounds included in said converting sound have been sounded in parallel from different sound sources, said converting spectrums having a plurality of peaks, a band of each of the peaks being wider than a corresponding band of each peak of spectrums of the input sound, and

wherein, as data representative of the converting spectrums acquired by said step of acquiring, said step of acquiring supplies, for each spectrum composing the converting spectrums, unit data composed of data indicative of a frequency and intensity of the spectrum at the frequency, and

wherein said step of generating includes;

a pitch conversion step of varying, in accordance with the pitch of the input sound detected by said step of detecting a pitch of an input sound, the frequency indicated by the unit data of each spectrum of the converting spectrums; and

an envelope adjustment step of adjusting a spectrum envelope of the converting spectrums, having the frequencies varied by said pitch conversion step, to substantially agree with the spectrum envelope of the input sound detected by said step of detecting a spectrum envelope of the input sound,

wherein, for each of the peaks in the converting spectrums, said envelope adjustment step

determines an intensity of a frequency in the spectrum envelope of the input sound, said frequency in the spectrum envelope of the input sound corresponding to the frequency of the peak in the converting spectrum,

determines a ratio between the determined intensity of the frequency in the spectrum envelope of the input sound and the intensity of a frequency in the converting spectrums as indicated by the unit data, said frequency in the converting spectrum corresponding to the peak in the converting spectrum, and

multiplies an intensity of the unit data of each spectrum belonging to the band centered around said peak in the converting spectrums, by the determined ratio,

wherein the spectrum envelope of the converting spectrums substantially agree with the spectrum envelope of the input sound.

14. A computer-readable medium containing a group of instructions for causing a computer to execute a procedure for processing an input sound, said procedure comprising:

a step of detecting a pitch of an input sound;

a step of detecting a spectrum envelope of the input sound;

a step of acquiring converting spectrums that is frequency spectrums of a converting sound comprising a plurality of sounds;

a step of generating output spectrums created by imparting the spectrum envelope of the input sound, detected by said step of detecting, to the converting spectrums acquired by said step of acquiring; and

a step of synthesizing a sound signal on the basis of the output spectrums generated by said step of generating,

wherein said plurality of sounds included in said converting sound have been sounded in parallel from different sound sources, said converting spectrums having a plurality of peaks, a band of each of the peaks being wider than a corresponding band of each peak of spectrums of the input sound, and

wherein, as data representative of the converting spectrums acquired by said step of acquiring, said step of acquiring supplies, for each spectrum composing the converting spectrums, unit data composed of data indicative of a frequency and intensity of the spectrum at the frequency, and

wherein said step of generating includes;

a pitch conversion step of varying, in accordance with the pitch of the input sound detected by said step of detecting a pitch of an input sound, the frequency indicated by the unit data of each spectrum of the converting spectrums; and

an envelope adjustment step of adjusting a spectrum envelope of the converting spectrums, having the frequencies varied by said pitch conversion step, to substantially agree with the spectrum envelope of the input sound detected by said step of detecting a spectrum envelope of the input sound,

wherein, for each of the peaks in the converting spectrums, said envelope adjustment step

determines an intensity of a frequency in the spectrum envelope of the input sound, said frequency in the spectrum envelope of the input sound corresponding to the frequency of the peak in the converting spectrum,

determines a ratio between the determined intensity of the frequency in the spectrum envelope of the input sound and the intensity of a frequency in the converting spectrums as indicated by the unit data, said frequency in the converting spectrum corresponding to the peak in the converting spectrum, and

multiplies an intensity of the unit data of each spectrum belonging to the band centered around said peak in the converting spectrums, by the determined ratio,

wherein the spectrum envelope of the converting spectrums substantially agree with the spectrum envelope of the input sound.

15. A method for processing an input sound, said method comprising:

a step of detecting a pitch of an input sound;

a step of detecting a spectrum envelope of the input sound;

a step of acquiring converting spectrums that are frequency spectrums of a converting sound;

a step of generating output spectrums created by imparting the spectrum envelope of the input sound, detected by said step of detecting, to the converting spectrums acquired by said step of acquiring; and

a step of synthesizing a sound signal on the basis of the output spectrums generated by said step of generating,

wherein said step of acquiring selectively acquires either one of first converting spectrums that are frequency spectrums of a first converting sound having a plurality of peaks, or second converting spectrums that are frequency spectrums of a second converting sound having substantially a same pitch as the first converting sound indicated by said first converting spectrums and having a plurality of peaks of a greater bandwidth at each peak than said first converting spectrums, at least said second converting sound comprising a plurality of sounds which have been sounded in parallel from different sound sources, a band of each of the peaks in at least said second converting spectrums being wider than a band of each peak of spectrums of the input sound, and

wherein, as data representative of the converting spectrums acquired by said step of acquiring, said step of acquiring supplies, for each spectrum composing the converting spectrums, unit data composed of data indicative of a frequency and intensity of the spectrum at the frequency, and

wherein said step of generating includes;

a pitch conversion step of varying, in accordance with the pitch of the input sound detected by said step of detecting a pitch of an input sound, the frequency indicated by the unit data of each spectrum of the converting spectrums supplied by said step of acquiring; and

an envelope adjustment step of adjusting a spectrum envelope of the converting spectrums, having the frequencies varied by said pitch conversion step, to substantially agree with the spectrum envelope of the input sound detected by said step of detecting a spectrum envelope of the input sound,

wherein, for each of the peaks in the converting spectrums, said envelope adjustment step

determines an intensity of a frequency in the spectrum envelope of the input sound, said frequency in the spectrum envelope of the input sound corresponding to the frequency of the peak in the converting spectrum,

determines a ratio between the determined intensity of the frequency in the spectrum envelope of the input sound and the intensity of a frequency in the converting spectrums as indicated by the unit data, said frequency in the converting spectrum corresponding to the peak in the converting spectrum, and

multiplies an intensity of the unit data of each spectrum belonging to the band centered around said peak in the converting spectrums, by the determined ratio,

wherein the spectrum envelope of the converting spectrums substantially agree with the spectrum envelope of the input sound.

16. A computer-readable medium containing a group of instructions for causing a computer to execute a procedure for processing an input sound, said procedure comprising:

a step of detecting a pitch of an input sound;

a step of detecting a spectrum envelope of the input sound;

a step of acquiring converting spectrums that are frequency spectrums of a converting sound;

a step of generating output spectrums created by imparting the spectrum envelope of the input sound, detected by said step of detecting, to the converting spectrums acquired by said step of acquiring; and

a step of synthesizing a sound signal on the basis of the output spectrums generated by said step of generating,

wherein said step of acquiring selectively acquires either one of first converting spectrums that are frequency spectrums of a first converting sound having a plurality of peaks, or second converting spectrums that are frequency spectrums of a second converting sound having substantially a same pitch as the first converting sound indicated by said first converting spectrums and having a plurality of peaks of a greater bandwidth at each peak than said first converting spectrums, at least said second converting sound comprising a plurality of sounds which have been sounded in parallel from different sound sources, a band of each of the peaks in at least said second converting spectrums being wider than a band of each peak of spectrums of the input sound, and

wherein, as data representative of the converting spectrums acquired by said step of acquiring, said step of acquiring supplies, for each spectrum composing the converting spectrums, unit data composed of data indicative of a frequency and intensity of the spectrum at the frequency, and

wherein said step of generating includes;

a pitch conversion step of varying, in accordance with the pitch of the input sound detected by said step of detecting a pitch of an input sound, the frequency indicated by the unit data of each spectrum of the converting spectrums supplied by said step of acquiring; and

an envelope adjustment step of adjusting a spectrum envelope of the converting spectrums, having the frequencies varied by said pitch conversion step, to substantially agree with the spectrum envelope of the input sound detected by said step of detecting a spectrum envelope of the input sound,

wherein, for each of the peaks in the converting spectrums, said envelope adjustment step

determines an intensity of a frequency in the spectrum envelope of the input sound, said frequency in the spectrum envelope of the input sound corresponding to the frequency of the peak in the converting spectrum,

determines a ratio between the determined intensity of the frequency in the spectrum envelope of the input sound and the intensity of a frequency in the converting spectrum as indicated by the unit data, said frequency in the converting spectrum corresponding to the peak in the converting spectrum, and

multiplies an intensity of the unit data of each spectrum belonging to the band centered around said peak in the converting spectrums, by the determined ratio,

wherein the spectrum envelope of the converting spectrums substantially agree with the spectrum envelope of the input sound.