Method and apparatus to model and transfer the prosody of tags across languages

• US 9,418,655 B2
• Filed: 01/17/2013
• Issued: 08/16/2016
• Est. Priority Date: 01/17/2013
• Status: Active Grant

• Current Assignee: Speech Morphing Systems, Inc.
• Original Assignee: Speech Morphing Systems, Inc.
• Inventors: Yassa, Fathy, Henton, Caroline
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US13/744,391
• Publication Number:
  US 20140200892A1
• Time in Patent Office: 1,307 Days
• Field of Search: 704/263, 704/277
• US Class Current: 1/1
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 2025/906 Pitch tracking

Abstract

A method of transferring the prosody of tag questions across languages includes extracting prosodic parameters of speech in a first language having a tag question and mapping the prosodic parameters to speech segments in a second language corresponding to the tag question. Accordingly, semantic and pragmatic intent of the tag question in the first language may be correctly conveyed in the second language.

Claims

1. A method to model and transfer the prosody of tag questions across languages, the method comprising:

receiving speech of a first person speaking in a first language;

analyzing the speech in the first language using automatic speech recognition;

extracting prosodic parameters of the speech in the first language and outputting text in the first language corresponding to the speech in the first language based on the analyzing;

searching the speech in the first language for a tag question in the first language;

translating the text in the first language to text in a second language;

outputting translated speech in the second language that is translated from the speech in the first language based on the translated text in the second language;

analyzing the speech in the first language to find speech segments that correspond to the tag question in the first language;

extracting a fundamental frequency from the speech segments that correspond to the tag question in the first language based on the extracted prosodic parameters of the speech in the first language;

fitting a stylized smooth contour to the fundamental frequency;

mapping the stylized smooth contour into a corresponding part of pitch range of the speech in the second language;

stretching or contracting the stylized smooth contour over time;

aligning the stylized smooth contour with corresponding speech segments in the second language that correspond to the tag question; and

applying the smooth contour to the speech in the second language.

2. The method of claim 1, wherein the stylized smooth contour is a cubic spline.

3. The method of claim 1, wherein the first language is English.

4. The method of claim 3, wherein the second language is a language other than English.

5. The method of claim 1, wherein the tag question is a grammatical structure in which a declarative statement or an imperative is turned into a question by adding an interrogative fragment.

6. The method of claim 1, wherein the prosodic parameters comprise at least one of pitch, pitch range, variability within the pitch range, and speech rate.