TECHNOLOGY FOR RESPONDING TO REMARKS USING SPEECH SYNTHESIS

• US 20160086597A1
• Filed: 06/02/2014
• Published: 03/24/2016
• Est. Priority Date: 05/31/2013
• Status: Active Grant

• Current Assignee: Yamaha Corporation
• Original Assignee: Yamaha Corporation
• Inventors: MATSUBARA, Hiroaki, URA, Junya, KAWAHARA, Takehiko, HISAMINATO, Yuji, YOSHIMURA, Katsuji
• Granted Patent: US 9,685,152 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G10L 13/027 Concept to speech synthesisers; Generation of natural phrases from machine-based concepts generation of parameters for speech synthesis out of text G10L13/08
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 13/0335 Pitch control
• CPC Class Codes: G10L 13/06 Elementary speech units used in speech synthesisers; Concatenation rules
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals
• CPC Class Codes: H04M 2201/39 using speech synthesis speech synthesis per se G10L13/00

Abstract

The present invention is provided with: a voice input section that receives a remark (a question) via a voice signal; a reply creation section that creates a voice sequence of a reply (response) to the remark; a pitch analysis section that analyzes the pitch of a first segment (e.g., word ending) of the remark; and a voice generation section (a voice synthesis section, etc.) that generates a reply, in the form of voice, represented by the voice sequence. The voice generation section controls the pitch of the entire reply in such a manner that the pitch of a second segment (e.g., word ending) of the reply assumes a predetermined pitch (e.g., five degrees down) with respect to the pitch of the first segment of the remark. Such arrangements can realize synthesis of replying voice capable of giving a natural feel to the user.

Claims

1. A voice synthesis apparatus comprising:

a voice input section configured to receive a voice signal of a remark;

a pitch analysis section configured to analyze a pitch of a first segment of the remark;

an acquisition section configured to acquire a reply to the remark; and

a voice generation section configured to generate voice of the reply acquired by said acquisition section, said voice generation section controlling a pitch of the voice of the reply in such a manner that a second segment of the reply has a pitch associated with the pitch of the first segment analyzed by said pitch analysis section.

2. The voice synthesis apparatus as claimed in claim 1, wherein the first segment is a word ending of the remark being a question, and said second segment is a word beginning or word ending of the reply.

3. The voice synthesis apparatus as claimed in claim 1, wherein said voice generation section controls the pitch of the voice of the reply in such a manner that an interval of the pitch of said second segment relative to the pitch of said first segment becomes a consonant interval except for a perfect unison.

4. The voice synthesis apparatus as claimed in claim 1, wherein said voice generation section controls the pitch of the voice of the reply in such a manner that an interval of the pitch of said second segment relative to the pitch of said first segment becomes any one of intervals, except for a perfect unison, within a range of one octave up and one octave down from the pitch of said first segment.

5. The voice synthesis apparatus as claimed in claim 3, wherein said voice generation section controls the pitch of the voice of the reply in such a manner that the interval of the pitch of said second segment relative to the pitch of said first segment becomes a consonant interval of five degrees down from the pitch of said first segment.

6. The voice synthesis apparatus as claimed in claim 3, wherein said voice generation section provisionally sets the pitch of the second segment of the voice of the reply at the pitch associated with the pitch of the first segment, and

said voice generation section is further configured to perform at least one of:

an operation of, if the provisionally-set pitch of the second segment is lower than a predetermined first threshold value, changing the provisionally-set pitch of the second segment to a pitch shifted one octave up; and

an operation of, if the provisionally-set pitch of the second segment is higher than a predetermined second threshold value, changing the provisionally-set pitch of the second segment to a pitch one octave down.

7. The voice synthesis apparatus as claimed in claim 3, wherein said voice generation section provisionally sets the pitch of the second segment of the voice of the reply at the pitch associated with the pitch of the first segment, and

said voice generation section is further configured to change the provisionally-set pitch to a pitch shifted one octave up or down in accordance with a designated attribute.

8. The voice synthesis apparatus as claimed in claim 1, wherein any one of a first mode and a second mode is settable as an operation mode of said voice generation section, and

wherein, in said first mode, said voice generation section controls the pitch of the voice of the reply in such a manner that the interval of the pitch of said second segment relative to the pitch of said first segment becomes a consonant interval except for a perfect unison, and

in said second mode, said voice generation section controls the pitch of the voice of the reply in such a manner that the interval of the pitch of said second segment relative to the pitch of said first segment becomes a dissonant interval.

9. The voice synthesis apparatus as claimed in claim 1, which further comprises:

a non-linguistic analysis section that analyzes non-linguistic information, other than pitch, related to the remark; and

a control section that controls voice generation, in said voice generation section, of the reply in accordance with the analyzed non-linguistic information.

10. The voice synthesis apparatus as claimed in claim 1, which further comprises:

a linguistic analysis section that analyzes linguistic information included in the remark and the reply; and

a control section that controls voice generation, in said voice generation section, of the reply in accordance with the analyzed linguistic information.

11. The voice synthesis apparatus as claimed in claim 1, which further comprises:

a non-linguistic analysis section that analyzes pitch variation in the remark; and

a control section that controls a pitch of voice of the reply, generated in said voice generation section, to vary in accordance with the pitch variation in the remark.

12. The voice synthesis apparatus as claimed in claim 1, wherein said voice generation section is configured to associate the pitch of said second segment with the pitch of said first segment in accordance with a given rule and generate voice with a characteristic based on a given agent attribute, and

which further comprises a control section that determines the rule based on at least one the agent attribute and an attribute of a speaker of the remark.

13. The voice synthesis apparatus as claimed in claim 1, wherein said voice generation section is configured to associate the pitch of said second segment with the pitch of said first segment in accordance with a given rule and generate voice with a characteristic based on a given agent attribute, and

which further comprises a control section that updates the rule based on receiving voice of a further remark via said sound input section after sounding of the voice of the reply.

14. A computer-implemented method comprising:

receiving a voice signal of a remark;

analyzing a pitch of a first segment of the remark;

acquiring a reply to the remark;

synthesizing voice of the acquired reply; and

controlling a pitch of the reply in such a manner that a pitch of a second segment of the voice of the reply has a pitch associated with the analyzed pitch of the first segment.

15. A non-transitory computer-readable storage medium containing a program comprising a group of instructions for causing a processor to perform:

receiving a voice signal of a remark;

analyzing a pitch of a first segment of the remark;

acquiring a reply to the remark;

synthesizing voice of the acquired reply; and

controlling a pitch of the reply in such a manner that a pitch of a second segment of the voice of the reply has a pitch associated with the analyzed pitch of the first segment.

16. A coding/decoding device comprising:

an A/D converter that converts an input voice signal of a remark into a digital signal;

a pitch analysis section that analyzes a pitch of a first segment of the remark based on the digital signal;

a back-channel feedback acquisition section that, when back-channel feedback is to be returned to the remark, acquires back-channel feedback data corresponding to a meaning of the remark;

a pitch control section that controls a pitch of the back-channel feedback data in such a manner that a second segment of the back-channel feedback data has a pitch associated with the analyzed pitch of the first segment; and

a D/A converter configured to convert the pitch-controlled back-channel feedback data into an analogue signal.

17. The coding/decoding device as claimed in claim 16, wherein the digital signal converted by said A/D converter is supplied to a host computer,

said pitch control section is further configured to receive replying voice data, responsive to the remark, returned from the host computer and control a pitch of the replying voice data in such a manner that a third segment of the received replying voice data has a pitch associated with the analyzed pitch of the first segment, and

said D/A converter is further configured to convert the pitch-controlled replying voice data, into an analogue signal.

18. A voice synthesis system comprising a coding/decoding device and a host computer,

said coding/decoding device comprising:

an A/D converter that converts an input voice signal of a remark into a digital signal;

a pitch analysis section that analyzes a pitch of a first segment of the remark based on the digital signal;

a back-channel feedback acquisition section that, when back-channel feedback is to be returned to the remark, acquires back-channel feedback data corresponding to a meaning of the remark;

a pitch control section that controls a pitch of the back-channel feedback data in such a manner that a second segment of the back-channel feedback data has a pitch associated with the analyzed pitch of the first segment; and

a D/A converter configured to convert the pitch-controlled back-channel feedback data into an analogue signal,

wherein said host computer is configured in such a manner that, when replying voice other than the back-channel feedback is to be returned to the remark, said host computer acquires replying voice data, responsive to the remark, in accordance with the digital signal converted by said A/D converter and returns the acquired replying voice data to said coding/decoding device,

said pitch control section is further configured to control a pitch of the replying voice data in such a manner that a third segment of the replying voice data returned from the host computer has a pitch associated with the analyzed pitch of the first segment, and

said D/A converter is further configured to convert the pitch-controlled replying voice data, into an analogue signal.

19. A method comprising:

converting, by means of an A/D converter, an input voice signal of a remark into a digital signal;

analyzing, by means of a processor, a pitch of a first segment of the remark based on the digital signal;

acquiring, by means of the processor, back-channel feedback data corresponding to a meaning of the remark, when back-channel feedback is to be returned to the remark;

controlling, by means of the processor, a pitch of the back-channel feedback data in such a manner that a second segment of the back-channel feedback data has a pitch associated with the analyzed pitch of the first segment; and

converting, by means of a D/A converter, the pitch-controlled back-channel feedback data into an analogue signal.

20. A non-transitory computer-readable storage medium containing a program for execution by a microprocessor provided in a coding/decoding device, said coding/decoding device including the microprocessor, an A/D converter that converts an input voice signal of a remark into a digital signal, and a D/A converter configured to convert the pitch-controlled back-channel feedback data into an analogue signal,

said program containing a group of instructions for causing the microprocessor to perform:

analyzing a pitch of a first segment of the remark based on the digital signal output from the A/D converter;

acquiring back-channel feedback data corresponding to a meaning of the remark, when back-channel feedback is to be returned to the remark; and

controlling a pitch of the back-channel feedback data in such a manner that a second segment of the acquired back-channel feedback data has a pitch associated with the analyzed pitch of the first segment, the pitch-controlled back-channel feedback data being supplied to the D/A converter.