System and method for prosodically modified unit selection databases

• US 10,249,290 B2
• Filed: 06/11/2018
• Issued: 04/02/2019
• Est. Priority Date: 05/12/2014
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Conkie, Alistair D., Golipour, Ladan, Syrdal, Ann K.
• Primary Examiner(s): Patel, Shreyans A
• Application Number:
  US16/004,812
• Publication Number:
  US 20180293972A1
• Time in Patent Office: 295 Days
• CPC Class Codes: G10L 13/06 Elementary speech units used in speech synthesisers; Concatenation rules
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 21/003 Changing voice quality, e.g. pitch or formants
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals
• CPC Class Codes: G11C 7/16 Storage of analogue signals in digital stores using an arrangement comprising analogue/digital [A/D] converters, digital memories and digital/analogue [D/A] converters

Abstract

Systems, methods, and computer-readable storage devices to improve the quality of synthetic speech generation. A system selects speech units from a speech unit database, the speech units corresponding to text to be converted to speech. The system identifies a desired prosodic curve of speech produced from the selected speech units, and also identifies an actual prosodic curve of the speech units. The selected speech units are modified such that a new prosodic curve of the modified speech units matches the desired prosodic curve. The system stores the modified speech units into the speech unit database for use in generating future speech, thereby increasing the prosodic coverage of the database with the expectation of improving the output quality.

Claims

1. A method comprising:

decomposing, via a residual-excited linear prediction algorithm, speech units from a speech unit database into residual coefficients and linear predictive coder coefficients;

determining a cost of modifying the residual coefficients to yield a determination;

modifying, via a pitch synchronous overlap and add algorithm, the residual coefficients, to yield modified residual coefficients based on the determination; and

combining, via the residual-excited linear prediction algorithm, the modified residual coefficients with the linear predictive coder coefficients, to yield new speech units, such that a new prosodic curve corresponding to the new speech units conforms to a desired prosodic curve, wherein speech is generated based on the new speech units.

2. The method claim 1, further comprising:

selecting, via a processor, the speech units from a speech unit database, where the speech units are used to generate speech correspond to text.

3. The method claim 1, further comprising:

identifying an actual prosodic curve of the speech units.

4. The method of claim 1, wherein the desired prosodic curve corresponds to a type of intonation.

5. The method of claim 4, wherein the type of intonation is one of a declarative intonation and a yes-no interrogative.

6. The method of claim 1, wherein the modifying of the residual coefficients comprises scaling a pitch of the residual coefficients.

7. The method of claim 1, wherein the cost of the modifying of the residual coefficients is higher than a cost of retrieving the new speech units from the speech unit database.

8. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

decomposing, via a residual-excited linear prediction algorithm, speech units from a speech unit database into residual coefficients and linear predictive coder coefficients;

determining a cost of modifying the residual coefficients to yield a determination;

modifying, via a pitch synchronous overlap and add algorithm, the residual coefficients, to yield modified residual coefficients based on the determination; and

combining, via the residual-excited linear prediction algorithm, the modified residual coefficients with the linear predictive coder coefficients, to yield new speech units, such that a new prosodic curve corresponding to the new speech units conforms to a desired prosodic curve, wherein speech is can be generated based on the new speech units.

9. The system claim 8, wherein the computer-readable storage medium stores additional instructions which, when executed by the processor, cause the processor to perform operations further comprising:

selecting, via a processor, the speech units from a speech unit database, where the speech units are used to generate speech correspond to text.

10. The system claim 8, wherein the computer-readable storage medium stores additional instructions which, when executed by the processor, cause the processor to perform operations further comprising:

identifying an actual prosodic curve of the speech units.

11. The system of claim 8, wherein the desired prosodic curve corresponds to a type of intonation.

12. The system of claim 11, wherein the type of intonation is one of a declarative intonation and a yes-no interrogative.

13. The system of claim 8, wherein the modifying of the residual coefficients comprises scaling a pitch of the residual coefficients.

14. The system of claim 8, wherein the cost of the modifying of the residual coefficients is higher than a cost of retrieving the new speech units from the speech unit database.

15. A non-transitory computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

decomposing, via a residual-excited linear prediction algorithm, speech units from a speech unit database into residual coefficients and linear predictive coder coefficients;

determining a cost of modifying the residual coefficients to yield a determination;

modifying, via a pitch synchronous overlap and add algorithm, the residual coefficients, to yield modified residual coefficients based on the determination; and

combining, via the residual-excited linear prediction algorithm, the modified residual coefficients with the linear predictive coder coefficients, to yield new speech units, such that a new prosodic curve corresponding to the new speech units conforms to a desired prosodic curve, wherein speech is generated based on the new speech units.

16. The non-transitory computer-readable storage device claim 15, wherein the non-transitory computer-readable storage device stores additional instructions stored which, when executed by the computing device, cause the computing device to perform operations further comprising:

selecting, via a processor, the speech units from a speech unit database, where the speech units are used to generate speech correspond to text.

17. The non-transitory computer-readable storage device claim 15, wherein the non-transitory computer-readable storage device stores additional instructions stored which, when executed by the computing device, cause the computing device to perform operations further comprising:

identifying an actual prosodic curve of the speech units.