Translation with conversational overlap

• US 9,747,282 B1
• Filed: 09/27/2016
• Issued: 08/29/2017
• Est. Priority Date: 09/27/2016
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Doppler Labs, Inc.
• Inventors: Baker, Jeffrey, Long, Paul Anthony, Garcia, Sal Gregory
• Primary Examiner(s): He, Jialong
• Application Number:
  US15/277,897
• Time in Patent Office: 336 Days
• CPC Class Codes: G06F 40/263 Language identification
• CPC Class Codes: G06F 40/47 Machine-assisted translation, e.g. using translation memory
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04R 1/1083 Reduction of ambient noise active noise reduction per se G10K11/175; protective devices for the ear, e.g. providing acoustic protection A61F11/06
• CPC Class Codes: H04R 2201/107 Monophonic and stereophonic headphones with microphone for two-way hands free communication
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

A first utterance is received. A first language associated with the first utterance is identified. The first utterance is translated into a second language and outputted at a pair of earphones that occludes a direct sound path associated with utterances.

Claims

1. A method, comprising:

receiving, at a device associated with a first user, a first utterance of a second user;

identifying a language of the second user based on the first utterance of the second user;

translating the first utterance of the second user from the language of the second user into a language of a first user;

receiving, at the device associated with the first user, a second utterance of the second user, wherein an utterance gap exists between the received first utterance of the second user and the received second utterance of the second user and the utterance gap is greater than a predetermined utterance gap and less than a turn threshold; and

outputting, at a pair of earphones, the translated first utterance of the second user, wherein a portion of the translated first utterance is outputted when the device associated with the first user is receiving the second utterance of the second user, wherein the pair of earphones occludes a direct sound path associated with spoken utterances.

2. The method of claim 1, wherein the first utterance is data associated with a spoken utterance transmitted from a second device.

3. The method of claim 1, wherein the first utterance is a spoken utterance and received via a microphone of the device associated with the first user.

4. The method of claim 1, wherein in the event the utterance gap exceeds the turn threshold the second user is determined to be finished speaking.

5. The method of claim 1, wherein outputting, at the pair of earphones, the translated first utterance associated with the second user includes outputting the translated first utterance of the second user at a first volume and outputting the received second utterance of the second user at a second volume, wherein the first volume of the translated first utterance of the second user is adjustable with respect to second volume of the second utterance of the second user.

6. The method of claim 1, further comprising:

receiving a first utterance of the first user at the device associated with the first user;

identifying the language of the first user based on the first utterance of the first user; and

transmitting data associated with the first utterance of the first user, wherein the data associated with the first utterance of the first user includes an identification of the language of the first user.

7. The method of claim 1, wherein one or more parameters associated with a microphone of the device associated with the first user are adjustable to focus in a direction of the first user.

8. The method of claim 1, wherein the pair of earphones occludes the direct sound path associated with spoken utterances by attenuating spoken utterances of the second user.

9. The method of claim 1, wherein the pair of earphones occludes the direct sound path associated with spoken utterances by attenuating spoken utterances of the second user, wherein an amount of attenuation associated with the spoken utterances of the second user is adjustable.

10. The method of claim 1, wherein in the event a translation of an utterance of the second user has a confidence less than a confidence threshold, the utterance is combined with one or more subsequent utterances to increase a confidence level of the combined utterance until a maximum condition is satisfied.

11. The method of claim 1, wherein the predetermined utterance gap is adjustable in the event the translated first utterance of the second user is less than a confidence threshold.

12. The method of claim 1, wherein in the event a translation of an utterance has a confidence less than a confidence threshold, the utterance is combined with a subsequent utterance, and the combined utterance is translated.

13. The method of claim 1, wherein the predetermined utterance gap is adjustable based at least in part on a speech pattern of the second user.

14. The method of claim 1, wherein the predetermined utterance gap is adjustable based at least in part on a cadence of the second user'"'"'s speech.

15. The method of claim 1, wherein a translated second utterance of the second user is outputted after the translated first utterance of the second user to match a duration between the first utterance of the second user and the second utterance of the second user.

16. The method of claim 1, wherein the pair of earphones outputs translated utterances to appear to come from a predetermined spatial location.

17. The method of claim 1, wherein the pair of earphones outputs translated utterances of the second user to appear to come from a spatial location of the second user.

18. A system comprising:

a communication interface configured to;

receive a first utterance of a second user; and

receive a second utterance of the second user, wherein an utterance gap exists between the received first utterance of the second user and the received second utterance of the second user and the utterance gap is greater than a predetermined utterance gap and less than a turn threshold;

one or more processors configured to;

identify a language of second user based on the first utterance of the second user; and

translate the first utterance of the second user from the language of the second user into a language of a first user; and

a pair of earphones configured to output the translated first utterance of the second user, wherein a portion of the translated first utterance is outputted when the communication interface is receiving the second utterance of the second user, wherein the pair of earphones occludes a direct sound path associated with spoken utterances.

19. The system of claim 18, wherein the first utterance is a spoken utterance or data associated with the spoken utterance.

20. A computer program product, the computer program product being embodied in a tangible non-transitory computer readable storage medium and comprising computer instructions for:

receiving a first utterance of a second user;

identifying a language of the second user based on the first utterance of the second user;

translating the first utterance of the second user from the language of the second user into a language of a first user;

receiving a second utterance of the second user, wherein an utterance gap exists between the received first utterance of the second user and the received second utterance of the second user and the utterance gap is greater than a predetermined utterance gap and less than a turn threshold; and

outputting, at a pair of earphones, the translated first utterance of the second user, wherein a portion of the translated first utterance is outputted when the second utterance of the second user is being received wherein the pair of earphones occludes a direct sound path associated with spoken utterances.