Auto-translation for multi user audio and video

• US 20150154183A1
• Filed: 12/12/2011
• Published: 06/04/2015
• Est. Priority Date: 12/12/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kristjansson, Trausti, Huang, John, Uszkoreit, Jakob David, Estelle, Joshua James, Buryak, Kirill, Konishi, Yusuke, Lin, Yu-Kuan, Tyan, Hung-ying, Wang, Chung-yi
• Granted Patent: US 9,110,891 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 2203/2061 Language aspects
• CPC Class Codes: H04M 2242/12 Language recognition, selection or translation arrangements
• CPC Class Codes: H04M 3/56 Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities video conference systems H04N7/15
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

The disclosed subject matter provides a system, computer readable storage medium, and a method providing an audio and textual transcript of a communication. A conferencing services may receive audio or audio visual signals from a plurality of different devices that receive voice communications from participants in a communication, such as a chat or teleconference. The audio signals representing voice (speech) communications input into respective different devices by the participants. A translation services server may receive over a separate communication channel the audio signals for translation into a second language. As managed by the translation services server, the audio signals may be converted into textual data. The textual data may be translated into text of different languages based the language preferences of the end user devices in the teleconference. The translated text may be further translated into audio signals.

Claims

1-39. -39. (canceled)

40. A computer-implemented method, comprising:

receiving, at a first end user device associated with a user, an audio data signal from a conferencing services server, the audio data signal representing a communication in a first spoken language received at a second end user device that is intended for the user;

determining, at the first end user device, the first spoken language of the communication represented by the audio data signal by receiving a user preferences signal, the user preferences signal comprising an identifier embedded in the audio data signal received from the conferencing services server;

determining, at the first end user device, language preferences of the user of the first end user device;

comparing, at the first end user device, the determined first spoken language with the language preferences of the user of the at the first end user device; and

when the determined first spoken language does not match the language preferences;

establishing, at the first end user device, a communication channel with a translation services server,

providing, from the first end user device, the audio data signal to the translation services server,

providing, from the first end user device, the language preferences of the user to the translation services server, and

receiving, at the first end user device, a translated audio data signal from the translation services server, the translated audio data signal representing the communication in a second spoken language corresponding to the language preferences and different from the first spoken language.

41. The method of claim 40, wherein determining the first spoken language of the communication represented by the audio data signal comprises recognizing the first spoken language of the communication.

44. The method of claim 40, further comprising, when the determined first spoken language does not match the language preferences, outputting, at the first end user device, an audio output corresponding to the translated audio data signal.

45. The method of claim 40, further comprising, when the determined first spoken language does match the language preferences, outputting, at the first end user device, an audio output corresponding to the audio data signal.

46. The method of claim 40, wherein the communication channel established with the translation services server is separate from a communication channel with the conferencing services server.

60. The method of claim 40, wherein the user preferences signal is based on language preferences of a second user associated with the second end user device.

42-43. -43. (canceled)

47. A computer program product having computer program code containing instructions embodied in a non-transitory machine readable storage medium, a processor executes the instructions to provide a method, the method comprising:

receiving an audio data signal from a conferencing services server, the audio data signal representing a communication in a first spoken language received at a second end user device that is intended for a user of a first end user device;

determining the first spoken language of the communication represented by the audio data signal by receiving a user preferences signal, the user preferences signal comprising an identifier embedded in the audio data signal received from the conferencing services server;

determining language preferences of the user of the first end user device;

comparing the determined first spoken language with the language preferences of the user of the at the first end user device; and

when the determined first spoken language does not match the language preferences;

establishing a communication channel with a translation services server,

providing the audio data signal to the translation services server,

providing the language preferences of the user to the translation services server, and

receiving a translated audio data signal from the translation services server, the translated audio data signal representing the communication in a second spoken language corresponding to the language preferences and different from the first spoken language.

48. The computer program product of claim 47, wherein determining the first spoken language of the communication represented by the audio data signal comprises recognizing the first spoken language of the communication.

50. The computer program product of claim 47, further comprising, when the determined first spoken language does not match the language preferences, outputting an audio output corresponding to the translated audio data signal.

51. The computer program product of claim 47, further comprising, when the determined first spoken language does match the language preferences, outputting an audio output corresponding to the audio data signal.

52. The computer program product of claim 47, wherein the communication channel established with the translation services server is separate from a communication channel with the conferencing services server.

61. The computer program product of claim 47, wherein the user preferences signal is based on language preferences of a second user associated with the second end user device.

49. (canceled)

53. A system, comprising:

one or more processors; and

a non-transitory machine readable storage medium storing code containing instructions that, when executed by the one or more processors, provide a method comprising;

receiving an audio data signal from a conferencing services server, the audio data signal representing a communication in a first spoken language received at a second end user device that is intended for a user of a first end user device;

determining the first spoken language of the communication represented by the audio data signal by receiving a user preferences signal, the user preferences signal comprising an identifier embedded in the audio data signal received from the conferencing services server;

determining language preferences of the user of the first end user device;

comparing the determined first spoken language with the language preferences of the user of the at the first end user device; and

when the determined first spoken language does not match the language preferences;

establishing a communication channel with a translation services server,

providing the audio data signal to the translation services server,

providing the language preferences of the user to the translation services server, and

receiving a translated audio data signal from the translation services server, the translated audio data signal representing the communication in a second spoken language corresponding to the language preferences and different from the first spoken language.

54. The system of claim 53, wherein determining the first spoken language of the communication represented by the audio data signal comprises recognizing the first spoken language of the communication.

57. The system of claim 53, further comprising, when the determined first spoken language does not match the language preferences, outputting an audio output corresponding to the translated audio data signal.

58. The system of claim 53, further comprising, when the determined first spoken language does match the language preferences, outputting an audio output corresponding to the audio data signal.

59. The system of claim 53, wherein the communication channel established with the translation services server is separate from a communication channel with the conferencing services server.

62. The system of claim 53, wherein the user preferences signal is based on language preferences of a second user associated with the second end user device.

55-56. -56. (canceled)