Method and apparatus for seamless transition of voice and/or text into sign language

• US 20040034522A1
• Filed: 08/14/2002
• Published: 02/19/2004
• Est. Priority Date: 08/14/2002
• Status: Active Grant

• Current Assignee: Alexander Trust
• Original Assignee: Alexander Nedzved, Alex Zapolsky, Anton Podoprigora, Biruk Oleg, Boris Nalibotski, Grigory Karapetian, Joshua E. Liebermann, Maureen Rosselli, Raanan Liebermann, Yuri Ilyich
• Inventors: Liebermann, Raanan, Nalibotski, Boris, Oleg, Biruk, Nedzved, Alexander, Podoprigora, Anton, Rosselli, Maureen, Zapolsky, Alex, Ilyich, Yuri, Karapetian, Grigory, Liebermann, Joshua E.
• Granted Patent: US 7,774,194 B2
• Time in Patent Office: Days
• US Class Current: 704/2
• CPC Class Codes: G06F 40/55 Rule-based translation
• CPC Class Codes: H04M 3/42391 where the subscribers are hearing-impaired persons, e.g. telephone devices for the deaf

Abstract

The present invention relates to a method and apparatus for seamlessly transitioning text into sign language. The apparatus comprises an encoding module and a translation module. The encoding module includes a camera for recording visual images of a model signing words and phrases, a video box for providing each of the images with a position code, and a processing unit with a memory for storing each of the recorded images so as to create a database of sign language video clips. The translation module includes a processing unit, a monitor associated with the processing unit for displaying a visual output in seamless sign language and a textual output, and a memory containing the database associated with the processing unit.

Claims

1. A method for creating a sign language database comprising the steps of:

recording an image of a model signing a sign for at least one word;

placing a video box having a plurality of numbered cells over said recorded image;

providing said recorded image with a code identification using said video box; and

storing each said recorded image and its respective code identification in a database.

2. A method according to claim 1, further comprising:

providing a device for initiating said recording step; and

initiating said recording step by activating said initiating device.

3. A method according to claim 2, wherein said initiating device providing step comprises providing a foot pedal and said recording step initiating step comprising said model depressing said foot pedal.

4. A method according to claim 1, wherein said recording step comprises capturing a live image of said model on a processing unit and said storing step comprises storing said captured image in a memory associated with said processing unit.

5. A method according to claim 1, further comprising playing back said recorded image and verifying that said sign is correct and entering a string identification for said recorded image.

6. A method according to claim 5, wherein said entering step comprises entering at least one word corresponding to said sign.

7. A method according to claim 5, wherein said code identification providing step comprises entering a four digit code representing initial and final positions of the right and left hands for the sign.

8. A method according to claim 1, further comprising:

storing each image on a storage device; and

invoking each said stored image from a directory box.

9. A method according to claim 1, further comprising generating a list of all transitional movements and identifying each of said transitional movements by a transitional code.

10. A method according to claim 9, further comprising performing each of said transitional movements using a live model who places his/her hands in the positions designated by said transition code assigned to a particular transitional movement and recording said performance of each said transitional movement.

11. A method according to claim 10, further comprising saving each said recorded performance in said database.

12. A method according to claim 1, wherein said image recording step comprises placing colored bands on the wrists of said model and capturing images of said colored bands.

13. A method according to claim 12, wherein said code identification providing step comprises performing a color segmentation analysis which calculates the coordinates of said colored bands and assigns a corresponding position code by determining the cells of the video box in which the bands are located.

14. An apparatus for creating a database containing a plurality of images representing the signing of words and phrases, said apparatus comprising:

means for recording visual images of a model signing said words and phrases;

means for providing each of said images with a position code; and

means for storing each of said recorded images.

15. An apparatus according to claim 14, further comprising means for allowing said model to view himself/herself as said visual images are being recorded.

16. An apparatus according to claim 14, further comprising means for inputting text corresponding to each of said recorded images.

17. An apparatus according to claim 14, wherein said storing means comprises a processing unit having a disk drive and a random accessible memory.

18. An apparatus according to claim 17, wherein said means for recording said images comprises a camera and said means for providing a position code comprises a numbered video box which may placed over each of said recorded images.

19. An apparatus according to claim 17, wherein said video box comprises a box divided into nine unequal rectangular cells.

20. An apparatus according to claim 17, further comprising means for displaying a dictionary of all words and phrases entered into said database.

21. A method for providing a user with a translation of an input into sign language comprising:

providing a translation module communicating with a database containing sign language video clips which form a dictionary;

inputting into said translation module an input containing at least one word;

converting said at least one inputted word into at least one of a textual output and a visual output in sign language; and

displaying at the user'"'"'s option one of said textual output and said visual output to said user.

22. A method according to claim 21, wherein said inputting step comprises inputting said at least one word from a microphone.

23. A method according to claim 22, further comprising:

passing said at least one word through a program for converting said at least one word for text; and

converting said textual version of said at least one word into one of a voice output and said visual output.

24. A method according to claim 23, further comprising lexically analyzing said at least one word for multiple meanings, idioms and phrases, names, number systems, money, words and sentences, misspellings, uncommon words, end of sentence, and auxiliaries.

25. A method according to claim 24, further comprising presenting multiple meanings of a word to said user and allowing said user to said select one of said meanings.

26. A method according to claim 24, further comprising forced finger spelling of a word having said multiple meanings.

27. A method according to claim 24, further comprising:

sequencing said at least one word for output;

retrieving video sign language clips from said database;

generating a seamless visual output display in sign language for said at least one word; and

displaying said seamless visual output display to said user.

28. A method according to claim 21, wherein said inputting step comprises inputting said at least one word from at least one of application software, the Internet, an e-mail, a close captioning device, and a text input device and converting said inputted at least one word into text.

29. A method according to claim 28, further comprising converting said text into at least one of a voice output and into a seamless visual output display in sign language.

30. A method according to claim 21, further comprising:

said inputting step comprising inputting a video; and

converting said video into a textual output.

31. A method according to claim 21, further comprising:

said inputting step comprising inputting a video; and

displaying said video to said user.

32. A method according to claim 21, further comprising:

inputting a musical input into said translational module; and

outputting vibrations representatives of said musical input.

33. An apparatus for providing a user with at least one of a textual output, a visual output in sign language, and a voice output, said apparatus comprising:

a processing unit;

a display monitor for presenting said textual output and said visual output in sign language connected to said processing unit;

means for generating a voice output associated with said processing unit; and

a dictionary stored in a memory, said dictionary containing video clips of signed words and phrases and said memory being accessible by said processing unit.

34. An apparatus according to claim 33, further comprising said monitor displaying a video output box, a textual box, and a dictionary box.

35. An apparatus according to claim 33, further comprising said processing unit being programmed to adjust the speed of said visual output.

36. An apparatus according to claim 33, further comprising said processing unit being programmed to adjust font size of said textual output.

37. An apparatus according to claim 33, further comprising said processing unit being programmed to eliminate and reinstate said dictionary.

38. An apparatus according to claim 33, further comprising said processing unit being programmed to eliminate and reinstate a box containing said textual output.

39. An apparatus according to claim 33, further comprising said processing unit being programmed to eliminate and reinstate a box containing said visual output.

40. An apparatus according to claim 33, further comprising said processing unit being programmed to remove and reinstate auxiliaries.

41. An apparatus according to claim 33, further comprising said processing unit being programmed to present misspelled words in said textual output in a color.

42. An apparatus according to claim 33, further comprising said processing unit being programmed to present words displayed as said visual output in color in a textual display.

43. An apparatus according to claim 33, further comprising said processing unit being programmed to display words having multiple meanings in a font different from other words being displayed.

44. An apparatus according to claim 33, further comprising means for inputting at least one of a textual input, a close-captioning input, an Internet input, an e-mail input, an application software input, a voice input, a video input, and a musical input.

45. An apparatus according to claim 44, further comprising means for dragging and dropping at least one of said inputs and displaying said at least one input on said monitor.

46. An apparatus according to claim 44, further comprising means for converting a musical input to a vibrational output.

47. An apparatus according to claim 44, further comprising means for converting said video input into text.

48. An apparatus according to claim 44, further comprising means for converting a voice input into text.

49. An apparatus according to claim 48, further comprising means for converting said text into at least one of said voice output and said visual output.

50. An apparatus according to claim 44, further comprising means for converting said application software input, said Internet input, said e-mail input, said close captioning input, and said textual input into at least one of said visual output and said voice output.

51. An apparatus according to claim 50, wherein said converting means comprises means for lexically analyzing said input, means for sequencing words for output, means for retrieving video clips from said dictionary representing said sequenced words, and means for generating a seamless visual signing of said words for display.