System and method for dual screen language translation

• US 9,501,472 B2
• Filed: 12/29/2012
• Issued: 11/22/2016
• Est. Priority Date: 12/29/2012
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Manuselis, Nicholas M., Valavi, John J.
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Sharma, Neeraj
• Application Number:
  US13/730,894
• Publication Number:
  US 20140188455A1
• Time in Patent Office: 1,424 Days
• Field of Search: 704/3, 704/277, 704/246, 704/271, 704/270, 704/2, 348/152, 348/738, 349/69, 455/566, 345/156, 345/173, 345/650, 379/202.01, 367/121, 381/92, 381/71.11, 381/302
• US Class Current: 1/1
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2499/11 Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04R 2499/15 Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops

Abstract

Generally, this disclosure provides systems and methods to facilitate real-time language translation between two speakers. A system may include an audio beamforming microphone configured to detect speech in a first language from a first speaker and to detect speech in a second language from a second speaker; a language translation module configured to translate the speech in the first language to text in the second language and to translate the speech in the second language to text in the first language; and a display element configured with a first display element side and a second display element side and further configured to display the text in the first language on the first display element side and to display the text in the second language on the second display element side.

Claims

1. A system to facilitate language translation, said system comprising:

a dual-sided display having a first display element and a second display element, said second display element opposite said first display element;

a first audio beamforming microphone corresponding to a first user and a second audio beamforming microphone corresponding to a second user, wherein each of said first and second audio beamforming microphones include a plurality of microphone elements to detect speech of said first user and said second user and each of said microphone elements generating a signal corresponding to said detected speech;

beamforming circuitry to combine each of said signals from said plurality of microphone elements corresponding to said first audio beamforming microphone into a first steerable audio beam and to combine each of said signals from said plurality of microphone elements corresponding to said second audio beamforming microphone into a second steerable audio beam, said beamforming circuitry forming each of said first and second steerable audio beams by introducing a phase shift into at least one of said generated signals forming each of said first and second steerable audio beams, wherein each phase shift is dynamically adjusted; and

language translation circuitry, wherein said language translation circuitry;

identifies a first voice pattern corresponding to speech of said first user in said first steerable audio beam;

identifies a second voice pattern corresponding to speech of said second user in said second steerable audio beam;

translates said speech of said first user from a first language to text in a second language;

translates said speech of said second user from said second language to text in said first language;

determines, based on said first and second steerable audio beams, whether said first display element or said second display element is displaying in a direction of said first user;

determines, based on said first and second steerable audio beams, whether said first display element or said second display element is displaying in a direction of said second user;

provides said text in said first language to said first display element when said first display element is determined to be displaying in said direction of said first user; and

provides said text in said second language to said second display element when said second display element is determined to be displaying in said direction of said second user.

2. The system of claim 1, wherein said first audio beamforming microphone is further configured to steer said first steerable audio beam in towards said first user and said second audio beamforming microphone is further configured to steer said second steerable audio beam in towards said second user.

3. The system of claim 2, wherein said first and said second steerable audio beams are to reduce background noise and improve language translation accuracy.

4. The system of claim 1, further comprising input circuitry to enable each user to configure said language translation circuitry by selecting said first and said second languages from a plurality of languages available for translation.

5. One or more non-transitory computer-readable memories having instructions stored thereon which when executed by a processor result in the following operations to facilitate language translation, said operations comprising:

detecting, using a first audio beamforming microphone, speech in a first language from a first user;

detecting, using a second audio beamforming microphone, speech in a second language from a second user, wherein said first and said second audio beamforming microphones include a plurality of microphone elements, each microphone element generating a signal corresponding to said detected speech;

combining each of said signals from said plurality of microphone elements corresponding to said first audio beamforming microphone into a first steerable audio beam;

combining each of said signals from said plurality of microphone elements corresponding to said second audio beamforming microphone into a second steerable audio beam, wherein each of said steerable audio beams is formed by introducing a phase shift into at least one of said generated signals forming each of said first and second steerable audio beams, wherein each phase shift is dynamically adjusted;

identifying a first voice pattern corresponding to said speech of said first user in said first steerable audio beam;

identifying a second voice pattern corresponding to said speech of said second user in said second steerable audio beam;

translating said speech in said first language to text in said second language;

translating said speech in said second language to text in said first language;

determining, based on said first and second steerable audio beams, whether a first display element or a second display element of a dual-sided display is displaying in a direction of said first user;

determining, based on said first and second steerable audio beams, whether said first display element or said second display element is displaying in a direction of said second user;

displaying said text in said first language on said first display element of said dual-sided display when it is determined, that said first display element is displaying in said direction of said first user; and

displaying said text in said second language on said second display element said dual-sided display when it is determined, that said second side is displaying in said direction of said second user, wherein said first display element is opposite said second display element.

6. The one or more non-transitory computer-readable memories of claim 5, further comprising the operation of steering said first steerable audio beam towards said first user and steering said second steerable audio beam towards said second user.

7. The one or more non-transitory computer-readable memories of claim 6, wherein said first and said second steerable audio beams reduce background noise and improve language translation accuracy.

8. The one or more non-transitory computer-readable memories of claim 5, further comprising the operation of selecting said first and said second languages from a plurality of languages available for translation.

9. A system comprising:

a processor;

input/output circuitry coupled to said processor;

a memory coupled to said processor;

a display coupled to said processor and having a first display element opposite a second display element;

a first and second audio beamforming microphone coupled to said processor and configured to detect speech in a first language from a first user and to detect speech in a second language from a second user, wherein said first and said second audio beamforming microphones include a plurality of microphone elements, each microphone element generating a signal corresponding to said detected speech;

beamforming circuitry to combine each of said signals from said plurality of microphone elements corresponding to said first audio beamforming microphone into a first steerable audio beam and to combine each of said signals from said plurality of microphone elements corresponding to said second audio beamforming microphone into a second steerable audio beam, said beamforming circuitry forming each of said steerable audio beams by introducing a phase shift into at least one of said generated signals forming each of said first and second steerable audio beams, wherein each phase shift is dynamically adjusted;

language translation circuitry coupled to said processor, said language translation circuitry to;

identify a first voice pattern corresponding to said speech of said first user in said first steerable audio beam;

identify a second voice pattern corresponding to said speech of said second user in said second steerable audio beam;

translate said speech of said first user from said first language to text in said second language;

translate said speech of said second user from said second language to text in said first language;

determine, based on said first and second steerable audio beams, whether said first display element or said second display element is displaying in a direction of said first user; and

determine, based on said first and second steerable audio beams, whether said first display element or said second display element is displaying in a direction of said second user, wherein said display is configured to;

display said text in said first language on said first display element when said first display element is displaying in said direction of said first user; and

display said text in said second language on said second display element when said second display element is displaying in said direction of said second user.

10. The system of claim 9, wherein said first and second audio beamforming microphones are further configured to steer said first steerable audio beam towards said first user and to steer said second steerable audio beam towards said second user.

11. The system of claim 10, wherein said first and said second steerable audio beams are to reduce background noise and improve language translation accuracy.

12. The system of claim 9, wherein said platform is selected from the group consisting of a laptop computing device, a tablet, a notebook and a smartphone.