METHODS EMPLOYING PHASE STATE ANALYSIS FOR USE IN SPEECH SYNTHESIS AND RECOGNITION

• US 20130226569A1
• Filed: 03/18/2013
• Published: 08/29/2013
• Est. Priority Date: 12/18/2008
• Status: Abandoned Application

• Current Assignee: Lessac Technologies, Inc.
• Original Assignee: Lessac Technologies, Inc.
• Inventors: CHANDRA, Nishant, NITISAROJ, Rattima, WILHELMS-TRICARICO, Reiner, MOTTERSHEAD, Brian, MARPLE, Gary A., REICHENBACH, John B.
• Application Number:
  US13/846,577
• Publication Number:
  US 20130226569A1
• Time in Patent Office: Days
• US Class Current: 704/219
• CPC Class Codes: G10L 13/027 Concept to speech synthesisers; Generation of natural phrases from machine-based concepts generation of parameters for speech synthesis out of text G10L13/08
• CPC Class Codes: G10L 13/06 Elementary speech units used in speech synthesisers; Concatenation rules
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation
• CPC Class Codes: G10L 15/1807 using prosody or stress
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

A computer-implemented method for automatically analyzing, predicting, and/or modifying acoustic units of prosodic human speech utterances for use in speech synthesis or speech recognition. Possible steps include: initiating analysis of acoustic wave data representing the human speech utterances, via the phase state of the acoustic wave data; using one or more phase state defined acoustic wave metrics as common elements for analyzing, and optionally modifying, pitch, amplitude, duration, and other measurable acoustic parameters of the acoustic wave data, at predetermined time intervals; analyzing acoustic wave data representing a selected acoustic unit to determine the phase state of the acoustic unit; and analyzing the acoustic wave data representing the selected acoustic unit to determine at least one acoustic parameter of the acoustic unit with reference to the determined phase state of the selected acoustic unit. Also included are systems for implementing the described and related methods.

Claims

1-10. -10. (canceled)

11. A method for categorically mapping the relationship of at least one text unit in a sequence of text to at least one corresponding prosodic phonetic unit, to at least one linguistic feature category in the sequence of text, and to at least one speech utterance represented in a synthesized speech signal, the method comprising:

(a) identifying, and optionally modifying, acoustic data representing the at least one speech utterance, to provide the synthesized speech signal;

(b) identifying, and optionally modifying, the acoustic data representing the at least one utterance to provide the at least one speech utterance with an expressive prosody determined according to prosodic rules; and

(c) identifying acoustic unit feature vectors for each of the at least one prosodic phonetic units, each acoustic unit feature vector comprising a bundle of feature values selected according to proximity to a statistical mean of the values of acoustic unit candidates available for matching with the respective prosodic phonetic unit and, optionally, for acoustic continuity with at least one adjacent acoustic feature vector.

12. A method according to claim 11 comprising selecting an acoustic unit for the sequence of acoustic units from the database of candidate acoustic units according to the proximity of the feature values of the selected acoustic unit to a corresponding acoustic feature vector on the acoustic unit pathway and, optionally, for acoustic continuity of the selected acoustic unit with the acoustic feature values of one or more neighboring acoustic units in the sequence of acoustic units.

13. A method according to claim 11 comprising selecting each acoustic unit for the sequence of acoustic units according to a rank ordering of acoustic units available to represent a specific prosodic phonetic unit in the sequence of text, the rank ordering being determined by the differences between the feature values of available acoustic units and the feature values of an acoustic feature vector on the acoustic unit pathway.

14. A method according to claim 11 comprising determining individual linguistic and acoustic weights for each prosodic phonetic unit according to linguistic feature hierarchies related to a prior adjacent prosodic phonetic unit and to a next adjacent prosodic phonetic unit, wherein each candidate acoustic unit can have different target and join weights for each respective end of the candidate acoustic unit.

15. A method according to claim 14 comprising:

(d) measuring one or more acoustic parameters, optionally F0, F1, F2, F3, energy, and the like, across a particular acoustic unit corresponding to a particular prosodic phonetic unit to determine time related changes in the one or more acoustic parameters; and

,

(e) modeling the particular acoustic unit and the relevant acoustic parameter values of the prior adjacent prosodic phonetic unit and the next adjacent prosodic phonetic unit.

16. A method according to claim 15 wherein the modeling comprises applying combinations of fourth-order polynomials and second- and third-order polynomials to represent n-dimensional trajectories of the modeled acoustic units through unconstrained acoustic space;

or comprises applying a lower-order polynomial to constrained acoustic space; and

optionally diphones and triphones.

17. A method according to claim 11 comprising using linguistically selected acoustical candidates for calculating acoustic features for synthesizing speech utterances from text by:

calculating an absolute and/or relative desired acoustic parameter value, optionally in terms of fundamental frequency and/or a change in fundamental frequency over the duration of the acoustic unit, the duration optionally being represented by a single point, multiple points, Hermite splines or another suitable representation, the desired acoustic parameter value being based on a weighted average of the actual acoustic parameters for a set of candidate acoustic units selected according to their correspondence to a particular linguistic context, wherein the weighting favors acoustic unit candidates more closely corresponding to the particular linguistic context.

18. A method for assigning linguistic and acoustic weights to prosodic phonetic units useful for concatenation into synthetic speech or for speech recognition, the method comprising:

determining individual linguistic and acoustic weights for each prosodic phonetic unit according to linguistic feature hierarchies related to a prior adjacent prosodic phonetic unit and to a next adjacent prosodic phonetic unit, wherein each candidate acoustic unit can have different target and join weights for each respective end of the candidate acoustic unit;

measuring one or more acoustic parameters, optionally F0, F1, F2, F3, energy, and the like, across a particular acoustic unit corresponding to a particular prosodic phonetic unit to determine time related changes in the one or more acoustic parameters; and

,

modeling the particular acoustic unit and the relevant acoustic parameter values of the prior adjacent prosodic phonetic unit and the next adjacent prosodic phonetic unit.

19. A method for deriving a path through acoustic space, the acoustic path comprising desired acoustic feature values for each sequential unit of a sequence of acoustic units to be employed in synthesizing speech from text, the method comprising:

calculating the acoustic path in absolute and/or relative coordinates, optionally in terms of fundamental frequency and/or a change in fundamental frequency over the duration of the synthesizing of the text, for the sequence of acoustic units, each desired sequential acoustic unit being represented by a representation, optionally a single point, multiple points, Hermite splines or another suitable acoustic unit representation, according to a weighted average of the acoustic parameters of the acoustic unit representation, wherein the weighted average is based on a degree of accuracy with which the acoustic parameters for each such sequentially desired acoustic unit are known, and on a degree of influence ascribed to each sequential acoustic unit according to the context of the acoustic unit in the sequence of desired acoustic units.

20-22. -22. (canceled)

23. A method of deriving an acoustic path comprising a sequence of desired acoustic units extending through unconstrained acoustic space, the acoustic path being useful for synthesizing speech from text with a desired style of speech prosody by concatenating the sequence of desired acoustic units, the method comprising:

(a) providing a database of acoustic units wherein each acoustic unit is identified according to a prosodic phonetic unit name and at least one additional linguistic feature; and

wherein each acoustic unit has been analyzed according to phase-state metrics so that pitch, energy, and spectral wave data can be modified simultaneously at one or more instants in time;

(b) mapping each acoustic unit to prosodic phonetic unit categorizations and additional linguistic categorizations enabling the acoustic unit to be specified and/or altered to provide one or more acoustic units for incorporation into expressively synthesized speech according to prosodic rules;

(c) calculating weighted absolute and/or relative acoustic values for a set of candidate acoustic units to match each desired acoustic unit, one candidate set per desired acoustic unit, matching being in terms of linguistic features for the corresponding mapped prosodic phonetic unit or a substitute for the corresponding mapped prosodic phonetic unit;

(d) calculating an acoustic path through n-dimensional acoustic space to be sequenced as an utterance of synthesized speech, the acoustic path being defined by the weighted average values for each candidate set of acoustic units;

(e) selecting and modifying, as needed, a sequence of acoustic units, or sub-units, for the synthesized speech according to the differences between the weighted acoustic values for a candidate acoustic unit, or sub-unit, and the weighted acoustic values of a point on the calculated acoustic path.

24-29. -29. (canceled)