Sign language telephone device

• US 6,477,239 B1
• Filed: 09/29/2000
• Issued: 11/05/2002
• Est. Priority Date: 08/30/1995
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Ikeda, Hisashi, Kaneko, Youichi, Hataoka, Nobuo, Sakiyama, Tomoko, Fujisawa, Hiromichi, Sagawa, Hirohiko, Ohki, Masaru
• Primary Examiner(s): Ramakrishnaiah, Melur
• Application Number:
  US09/675,052
• Time in Patent Office: 767 Days
• Field of Search: 379/52, 379/93.15, 348/14.01, 348/14.02, 348/14.03, 348/14.05, 348/14.06, 348/14.07, 348/14.08, 348/14.04, 340/825.19, 382/100, 704/2, 704/3, 704/271, 704/272
• US Class Current: 379/52
• CPC Class Codes: G09B 21/009 Teaching or communicating with deaf persons G09B21/04 takes precedence; deaf and dumb persons G09B21/00
• CPC Class Codes: H04M 11/066 Telephone sets adapted for data transmision
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/440236 by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/478 Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 5/45 Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

A sign language telephone device is offered which enables an aurally handicapped person who uses the sign language to converse with a normal person at a distant place who does not know the sign language. The sign language telephone device is placed on the side of the aurally handicapped person, and hand gestures of the sign language inputted from a sign language input means are recognized as the sign language, and the recognized sign language is translated to Japanese. The translated Japanese word train is converted to synthesized voices and it is transmitted to a videophone on the side of a normal person. The voices from the videophone are recognized, and the recognized Japanese is translated to the sign language to generate sign language animations and they are displayed on the screen of a TV set on the side of the aurally handicapped person. According to the present invention, it is made possible for an aurally handicapped person to have conversation easily with a normal person at a distant place who does not know the sign language through an existing network.

Claims

1. A sign language telephone device, comprising:

a videophone device;

sign language telephone control device operatively connected to said videophone device via a network, which translates an input sign language, generates synthesized voices for transmission to said videophone device via said network, recognizes voices from said videophone device, and generates a sign language animation from the recognized results for a visual display, said sign language telephone control device comprising;

sign-language-to-text language-translation subsystem comprising;

a sign language recognition unit which recognizes an inputted sign language in a first language to a sign language words train in said first language;

a sign-language-to-text language-translation unit which translates the recognized sign language words train in said first language to a text language words train in said first language; and

a voice synthesizing unit which synthesizes the translated text language words train as voices;

a text language-to-sign-language-translation subsystem comprising a voice recognition unit which recognizes the voices received from said videophone device and translates the recognized voices into a sign language words train; and

a gesture to sign language animation conversion unit which generates animations by using positional data of gestures in said inputted sign language.

2. A sign language telephone device according to claim 1, further comprising:

a gesture to sign language animation conversion unit which generates animations by using positional data of gestures in said inputted sign language.

3. A sign language telephone device comprising:

a videophone device;

a sign language telephone control device operatively connected to said videophone device via a network, which translates an input sign language, generates synthesized voices for transmission to said videophone device via said network, recognizes voices from said videophone device, and generates a sign language words train from the recognized results for a visual display, said sign language telephone control device comprising;

a sign-language-to-text language translation subsystem comprising;

a sign language recognition unit which recognizes an inputted sign language in a first language to a sign language words train in said first language;

a sign-language-to-text language translation unit which translates the recognized sign language words train in said first language to a text language words train in said first language; and

a voice synthesizing unit which synthesizes the translated text language words train as voices;

a text language-to-sign-language-translation subsystem comprising a voice recognition unit which recognizes the voices received from said videophone device and translates the recognized voices into a sign language words train; and

said videophone device comprising a text language-translation subsystem which translates the received first language words train into a second language words train; and

a gesture to sign language animation conversion unit which generates animations by using positional data of gestures in said inputted sign language.

4. A sign language telephone device comprising:

a videophone device;

a sign language telephone control device operatively connected to said videophone device via a network, which translates an input sign language, generates synthesized voices for transmission to said videophone device via said network, recognizes voices from said videophone device, and generates a sign language words train from the recognized results for a visual display, said sign language telephone control device comprising;

a sign-language-to-text language-translation subsystem comprising;

a sign language recognition unit which recognizes an inputted sign language in a first language to a sign language words train in said first language;

a sign-language-to-text language-translation unit which translates the recognized sign language words train in said first language to a text language words train in said first language; and

a voice synthesizing unit which synthesizes the translated text language words train as voices;

a text language-to-sign-language-translation subsystem comprising a voice recognition unit which recognizes the voices received from said videophone device and translates the recognized voices into a sign language words train; and

said videophone device comprising a text language-translation subsystem which translates a sending language in a first language into a second language; and

a gesture to sign language animation conversion unit which generates animations by using positional data of gestures in said inputted sign language.

5. A sign language telephone device comprising:

a videophone device;

a sign language telephone control device operatively connected to said videophone device via a network, which translates an input sign language, generates synthesized voices for transmission to said videophone device via said network, recognizes voices from said videophone device, and generates a sign language words train from the recognized results for a visual display, said sign language telephone control device comprising;

a sign-language-to-text language-translation subsystem comprising;

a sign language recognition unit which recognizes an inputted sign language in a first language to a sign language words train in said first language;

a sign-language-to-text language-translation unit which translates the recognized sign language words train in said first language to a text language words train in said first language; and

a voice synthesizing unit synthesizing the translated text language words train as voices; and

a text language-to-sign-language-translation subsystem comprising a voice recognition unit which recognizes the voices in at least one of said first language and a second language received from said videophone device, and translates the recognized voices into a sign language words train; and

a gesture to sign language animation conversion unit which generates animations by using positional data of gestures in said inputted sign language.

6. A sign language telephone device connected to a videophone device via a network, said sign language telephone device comprising:

a display device; and

a sign language telephone control device operatively connected to said videophone device via a network, which translates an input sign language, generates synthesized voices for transmission to said videophone device via said network, recognizes voices from said videophone device, and generates a sign language words train from the recognized results for a visual display, said sign language telephone control device comprising;

means for displaying an image received from said videophone device on said display device;

a sign-language-to-text language-translation subsystem for performing translation of an inputted sign language to a text language, said sign-language-to-text language-translation subsystem comprising;

a sign language recognition unit which recognizes the inputted sign language in a first language to a sign language words train in said first language;

a sign-language-to-text language-translation unit which translates the recognized sign language words train in said first language to a text language words train in at least one of said first language and a second language, and

a voice synthesizing unit which synthesizes the translated text language words train as voices;

a text language-to-sign-language-translation subsystem for performing translation of a text language to a language, said text language-to-sign-language-translation subsystem comprising;

a voice recognition unit which recognizes the voices received from said videophone device, as a text language words train;

a text language-to-sign-language-translation unit which translates said text language words train recognized in said voice recognition unit, into a sign language words train;

means for displaying said sign language words train which is translated from said recognized voices from said videophone device, simultaneously with the image received from said videophone device, on said display device;

means for displaying said sign language words train which is translated from the recognized sign language in said first language inputted through said sign language telephone device, on said display device, with the image received from said videophone device; and

means for displaying at least one of said text language words train which was translated in said sign-language-to-text language-translation unit, and said text language words train which was recognized in said voice recognition unit, on said display device, with the image received from said videophone device, said text language words train which was translated in said sign-language-to-text language-translation unit being at least one of said first language and said second language.

7. A videophone device connected to a sign language telephone device via a network, said sign language telephone device comprising:

a sign language telephone control device operatively connected to said videophone device via a network, which translates an input sign language, generates synthesized voices for transmission to said videophone device via a network, recognizes voices from said videophone device, and generates a sign language words train from the recognized results for a visual display, said sign language telephone control device comprising;

a sign-language-to-text language-translation subsystem comprising;

a sign language recognition unit which recognizes an inputted sign language in a first language to a sign language words train in said first language;

a sign-language-to-text language-translation unit which translates the recognized sign language words train in said first language to a text language words train in at least one of said first language and a second language;

a voice synthesizing unit which synthesizes the translated text language words train as voices;

a text language-to-sign-language-translation subsystem comprising a voice recognition unit which recognizes the voices received from said videophone device and translates the recognized voices into a sign language words train;

said videophone device comprising;

a display device;

means for displaying an image received from said sign language telephone device, on said display device;

means for displaying said text language words train in at least one of said first language and said second language translated in said sign-language-to-text language-translation unit, simultaneously with the image received from said sign language telephone device, on said display device.

8. A videophone device according to claim 7, wherein at least one of said videophone device and said sign language telephone device comprises:

a text language-translation unit which translates the sign language words train which was recognized from said voices received from said videophone device, to a text language words train; and

means for displaying said text language words train translated in said text language-translation unit, simultaneously with the image received from said sign language telephone device, on said display device.

9. A videophone device according to claim 7, wherein said image received from said sign language telephone device and displayed on said display device, is a sign language words train recognized from said inputted sign language.

10. A videophone device according to claim 8, wherein said image received from said sign language telephone device and displayed on said display device, is a sign language words train recognized from said inputted sign language.