Hybrid, offline/online speech translation system
First Claim
1. A speech translation system comprising:
- a translation server; and
a client device that is configured for communicating with the translation server, wherein the client device comprises;
a microphone;
a processor connected to the microphone;
a memory connected to the processor that stores instructions to be executed by the processor; and
a speaker connected to the processor,wherein;
the client device is for outputting, via the speaker, a translation of verbally input phrases from a first language to a second language; and
the memory stores instructions such that;
the processor determines the second language for verbally input phrases received at the client device from a user of the client device;
the processor receives from the user a translation mode setting for the client device for the translation of the verbally input phrase into the determined second language, the translation mode setting comprising a privacy preference of using the translation server only if a secure wireless network is available;
in response to determining that a secure wireless network is not available, the translation is automatically selected to be performed at the client device, the translation comprising;
translating the verbally input phrases from the first language into the second language; and
outputting, to the user in the second language, a local translation of the verbally input phrases;
in response to determining that a secure wireless network is available, the translation is automatically selected to be performed at the translation server, the translation comprising;
the client device sending, to the translation server, information associated with the input verbally phrases in the first language received by the client device;
the translation server determining a server translation of the verbally input phrases in the second language based on the data received via the wireless network from the client device; and
the translation server transmitting, to the client device, data regarding the server translation of the verbally input phrases in the second language, such that the client device outputs the server translation;
the translation server monitors, over time, speech utterances received by the client device for translation from the first language to the second language;
the translation server determines, based on the monitored speech utterances, vocabulary used by the user; and
the translation server updates, based on the determined vocabulary, at least one of the local acoustic model, the local language model, the local translation model and the local speech synthesis model of the client device, wherein updates to the at least one of the local acoustic model, the local language model, the local translation model and the local speech synthesis model of the client device are transmitted from the translation server to the client device via the wireless network.
3 Assignments
0 Petitions
Accused Products
Abstract
A hybrid speech translation system whereby a wireless-enabled client computing device can, in an offline mode, translate input speech utterances from one language to another locally, and also, in an online mode when there is wireless network connectivity, have a remote computer perform the translation and transmit it back to the client computing device via the wireless network for audible outputting by client computing device. The user of the client computing device can transition between modes or the transition can be automatic based on user preferences or settings. The back-end speech translation server system can adapt the various recognition and translation models used by the client computing device in the offline mode based on analysis of user data over time, to thereby configure the client computing device with scaled-down, yet more efficient and faster, models than the back-end speech translation server system, while still be adapted for the user'"'"'s domain.
-
Citations
15 Claims
-
1. A speech translation system comprising:
-
a translation server; and a client device that is configured for communicating with the translation server, wherein the client device comprises; a microphone; a processor connected to the microphone; a memory connected to the processor that stores instructions to be executed by the processor; and a speaker connected to the processor, wherein; the client device is for outputting, via the speaker, a translation of verbally input phrases from a first language to a second language; and the memory stores instructions such that; the processor determines the second language for verbally input phrases received at the client device from a user of the client device; the processor receives from the user a translation mode setting for the client device for the translation of the verbally input phrase into the determined second language, the translation mode setting comprising a privacy preference of using the translation server only if a secure wireless network is available; in response to determining that a secure wireless network is not available, the translation is automatically selected to be performed at the client device, the translation comprising; translating the verbally input phrases from the first language into the second language; and outputting, to the user in the second language, a local translation of the verbally input phrases; in response to determining that a secure wireless network is available, the translation is automatically selected to be performed at the translation server, the translation comprising; the client device sending, to the translation server, information associated with the input verbally phrases in the first language received by the client device; the translation server determining a server translation of the verbally input phrases in the second language based on the data received via the wireless network from the client device; and the translation server transmitting, to the client device, data regarding the server translation of the verbally input phrases in the second language, such that the client device outputs the server translation; the translation server monitors, over time, speech utterances received by the client device for translation from the first language to the second language; the translation server determines, based on the monitored speech utterances, vocabulary used by the user; and the translation server updates, based on the determined vocabulary, at least one of the local acoustic model, the local language model, the local translation model and the local speech synthesis model of the client device, wherein updates to the at least one of the local acoustic model, the local language model, the local translation model and the local speech synthesis model of the client device are transmitted from the translation server to the client device via the wireless network. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A method comprising:
-
receiving at a client device, from a user of the client device, a verbally input phrase in a first language; determining a second language for translation of the verbally input phrase; receiving from the user a translation mode setting for the client device for the translation of the verbally input phrase into the determined second language, the translation mode setting comprising a privacy preference of using the translation server only if a secure wireless network is available; in response to determining that the a secure wireless network is not available, automatically selecting to perform the translation at the client device, the translation comprising; translating, by the client device, the verbally input phrase from the first language into the second language; and outputting, in the second language, a local translation of the verbally input phrase; in response to determining that the a secure wireless network is available, automatically selecting to perform the translation at a translation server, the translation comprising; sending, from the client device to the translation server, information associated with the verbally input phrase; receiving, at the client device from the translation server, data associated with a sever translation of the verbally input phrase from the first language to the second language; and outputting, in the second language, the server translation of the verbally input phrase; monitoring, over time, speech utterances received by the client device for translation from the first language to the second language; determining, based on the monitored speech utterances, vocabulary used by the user; and updating, based on the determined vocabulary, at least one of a local acoustic model, a local language model, a local translation model and a local speech synthesis model of the client device, wherein updates to the at least one of the local acoustic model, the local language model, the local translation model and the local speech synthesis model of the client device are transmitted from the translation server to the client device via the wireless network. - View Dependent Claims (8, 9, 10, 11, 12, 13, 14, 15)
-
Specification