Translation with conversational overlap

• US 10,437,934 B2
• Filed: 07/14/2017
• Issued: 10/08/2019
• Est. Priority Date: 09/27/2016
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Baker, Jeffrey, Garcia, Sal Gregory, Long, Paul Anthony
• Primary Examiner(s): He, Jialong
• Application Number:
  US15/650,561
• Publication Number:
  US 20180089179A1
• Time in Patent Office: 816 Days
• CPC Class Codes: G06F 40/263 Language identification
• CPC Class Codes: G06F 40/47 Machine-assisted translation, e.g. using translation memory
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04R 1/1083 Reduction of ambient noise active noise reduction per se G10K11/175; protective devices for the ear, e.g. providing acoustic protection A61F11/06
• CPC Class Codes: H04R 2201/107 Monophonic and stereophonic headphones with microphone for two-way hands free communication
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

A plurality of utterances of a first user from the language of the first user is translated into a language of a second user. The confidence scores associated with the translated utterances are compared with a confidence threshold. A predetermined utterance gap is adjusted based on the comparison. The predetermined utterance gap is a duration of time that occurs between utterances.

Claims

1. A method, comprising:

receiving a plurality of utterances of a first person in a first language;

detecting an utterance gap between sequential utterances of the plurality of utterances;

determining, prior to translating an utterance, whether the utterance will be translated by comparing the utterance gap after the utterance is completed to a threshold utterance gap and, if it is determined that the utterance will be translated;

translating the utterance from the first language to a second language to produce a translated utterance;

determining a translation confidence score for the translated utterance;

determining whether the confidence score is greater than or equal to a confidence level;

determining, based on whether the confidence score is greater than or equal to the confidence level, whether the confidence score is great enough to output the translated utterance; and

determining accrued translation confidence scores for a plurality of utterances, wherein the threshold utterance gap is increased if a percentage of the accrued translation confidence scores is less than a confidence threshold.

2. The method of claim 1, wherein the plurality of utterances include a first utterance of the first person and a second utterance of the first person, wherein the first utterance of the first person and the second utterance of the first person are received at a device associated with second person.

3. The method of claim 2, wherein the first utterance of the first person and the second utterance of the first person are data associated with spoken utterances transmitted from a device associated with the first person.

4. The method of claim 2, wherein the first utterance of the first person and the second utterance of the first person are spoken utterances of the first person.

5. The method of claim 1, wherein the threshold utterance gap is less than a turn threshold duration.

6. The method of claim 1, further comprising outputting the translated utterance at a device associated with a second person.

7. The method of claim 6, wherein the device associated with the second person includes a pair of earbuds.

8. The method of claim 7, wherein the pair of earbuds are configured to occlude a direct sound path associated with the plurality of utterances of the first person by attenuating the plurality of utterances.

9. The method of claim 8, wherein an amount of attenuation of the plurality of utterances is adjustable.

10. The method of claim 6, wherein the translated utterance is outputted to appear to come from a predetermined spatial location.

11. The method of claim 6, wherein the translated utterance is outputted to appear to come from a spatial location of the first person.

12. The method of claim 1, wherein the threshold utterance gap is adjustable based at least in part on a speech pattern of the first person.

13. The method of claim 1, wherein the threshold utterance gap is adjustable based at least in part on a cadence of the first person'"'"'s speech.

14. A system, comprising:

a processor configured for;

receiving a plurality of utterances of a first person in a first language;

detecting an utterance gap between sequential utterances of the plurality of utterances;

determining, prior to translating an utterance, whether the utterance will be translated by comparing the utterance gap after the utterance is completed to a threshold utterance gap and, if the processor determines that the utterance will be translated;

translating the utterance from the first language to a second language to produce a translated utterance;

determining a translation confidence score for the translated utterance;

determining whether the confidence score is greater than or equal to a confidence level;

determining, based on whether the confidence score is greater than or equal to the confidence level, whether the confidence score is great enough to output the translated utterance; and

determining accrued translation confidence scores for a plurality of utterances, wherein the threshold utterance gap is decreased if a percentage of the accrued translation confidence scores is greater than or equal to a confidence threshold and wherein the confidence threshold corresponds with a percentage of translations that are accurate.

15. The system of claim 14, wherein the threshold utterance gap is increased if a percentage of the accrued translation confidence scores is less than the confidence threshold.

16. The system of claim 14, wherein the processor is further configured to output the translated plurality of utterances at a device associated with a second person.

18. The system of claim 14, wherein the processor is further configured for determining whether a maximum condition is satisfied if the processor determines that the confidence score is not great enough to output the translated utterance.

19. The system of claim 18, wherein the processor is further configured for combining the translated utterance with a subsequent utterance if the processor determines that the maximum condition is not satisfied.

17. A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

receiving a plurality of utterances of a first person in a first language;

detecting an utterance gap between sequential utterances of the plurality of utterances;

determining, prior to translating an utterance, whether the utterance will be translated by comparing the utterance gap after the utterance is completed to a threshold utterance gap and, if it is determined that the utterance will be translated;

translating the utterance from the first language to a second language to produce a translated utterance;

determining a translation confidence score for the translated utterance;

determining whether the confidence score is greater than or equal to a confidence level;

determining, based on whether the confidence score is greater than or equal to the confidence level, whether the confidence score is great enough to output the translated utterance; and

determining accrued translation confidence scores for a plurality of utterances, wherein the threshold utterance gap is increased if a percentage of the accrued translation confidence scores is less than a confidence threshold.