Matching, synchronization, and superposition on orginal speaking subject images of modified signs from sign language database corresponding to recognized speech segments
First Claim
Patent Images
1. A method of generating a synthesized signing image comprising the steps of:
- storing a sign language database including a sign image corresponding to each of a plurality of speech segments;
obtaining an original image of a subject;
identifying a characteristic of the subject in the original image;
receiving a speech segment spoken by the subject;
identifying one of said database sign images corresponding to the speech segment spoken by the subject;
modifying the identified sign image to match the characteristic of the subject in the original image; and
superimposing the modified sign image on the original image of the subject to generate a synthesized signing image.
4 Assignments
0 Petitions
Accused Products
Abstract
A synthesized signing image is generated which includes an original image of a subject and a sign image superimposed on the original subject image. The sign image is selected from a sign language database based on the speech (i.e., a letter or word) spoken by the subject. Based on the size and tone (intensity) of the subject'"'"'s face in the original image, the database sign image is modified to match the intensity and have a proportional size to the subject'"'"'s face. The modified database sign image is then superimposed on the original image of the subject to create a synthesized image in which the subject appears to be signing.
-
Citations
22 Claims
-
1. A method of generating a synthesized signing image comprising the steps of:
-
storing a sign language database including a sign image corresponding to each of a plurality of speech segments; obtaining an original image of a subject; identifying a characteristic of the subject in the original image; receiving a speech segment spoken by the subject; identifying one of said database sign images corresponding to the speech segment spoken by the subject; modifying the identified sign image to match the characteristic of the subject in the original image; and superimposing the modified sign image on the original image of the subject to generate a synthesized signing image. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A method of generating a synthesized signing image comprising the steps of:
-
storing a sign language database including a sign image corresponding to each of a plurality of speech segments, each said sign image comprising one or more signing hands; obtaining an original image of a subject, said subject including a face; calculating the relative size of the subject'"'"'s face in the original image; calculating the relative size of the one or more hands in the sign image; receiving a speech segment spoken by the subject; identifying one of said database sign images corresponding to the speech segment spoken by the subject; modifying the size of the hands in the identified sign image to be a predetermined proportion to the size of the subject'"'"'s face in the original image; and superimposing the modified sign image on the original image of the subject to generate a synthesized signing image.
-
-
14. A method of generating a synthesized signing image comprising the steps of:
-
storing a sign language database including a sign image corresponding to each of a plurality of speech segments, each said sign image comprising one or more signing hands having a natural variation in intensity due to the shape of the hands; obtaining an original image of a subject, said subject including a face; calculating the average intensity of the subject'"'"'s face; calculating the average intensity of the hands in the identified sign image; receiving a speech segment spoken by the subject; identifying one of said database sign images corresponding to the speech segment spoken by the subject; modifying the intensity of the hands in the identified sign image to generally match the intensity of the subject'"'"'s face in the original image, while maintaining a variation in the intensity of the hands in the modified sign image which is similar to the variation in the intensity of the hands in the database sign image; and superimposing the modified sign image on the original image of the subject to generate a synthesized signing image.
-
-
15. A method of generating a synthesized signing image comprising the steps of:
-
storing a sign language database at both a transmitting system and a receiving system, said sign language database including a sign image corresponding to each of a plurality of speech segments and a sign image code corresponding to each sign image; performing the following steps at the transmitter system; a) obtaining an original image of a subject; receiving a speech segment spoken by the subject; b) identifying one of said database sign images corresponding to the speech segment spoken by the subject; c) transmitting the original image and the sign image code corresponding to the identified sign image; performing the following steps at the receiving system; d) receiving the transmitted original image and the sign image code corresponding to the identified sign image; e) identifying a characteristic of the subject in the received original image; f) modifying the sign image corresponding to the received sign image code to match the characteristic of the subject in the received original image; and g) superimposing the modified sign image on the original image of the subject to generate a synthesized signing image.
-
-
16. A method of processing an original image to facilitate the generation of a synthesized signing image:
-
a) storing a sign language database including a sign image corresponding to each of a plurality of speech segments and a sign image code corresponding to each sign image; b) obtaining an original image of a subject; c) receiving a speech segment spoken by the subject, said speech segment corresponding to the original image; d) identifying one of said database sign images corresponding to the speech segment spoken by the subject; e) measuring one or more characteristics of the original image; f) measuring one or more characteristics of the identified database sign image; g) calculating, based on said measured characteristics of the original image and the sign image, one or more modification parameters indicating the amount which the sign image should be modified to approximately match the one or more measured characteristics of the original image; and h) transmitting to a remote location a signal comprising the sign image code corresponding to the identified sign image, the one or more calculated modification parameters, and the original image, thereby to facilitate the generation of a synthesized signing image at the remote location. - View Dependent Claims (17, 18)
-
-
19. A method of generating a synthesized signing image comprising the steps of:
-
storing a sign language database including a sign image corresponding to each of a plurality of speech segments and a sign image code corresponding to each sign image; receiving a signal including an original image of a subject and a sign image code corresponding to a database sign image, and one or more modification parameters; modifying the database sign image corresponding to the received sign image code based on the received one or more modification parameters; superimposing the modified sign image on the original image of the subject to generate a synthesized signing image.
-
-
20. An encoding system for encoding a signal to facilitate the generation of a synthesized signing image at a remote location, comprising:
-
a memory storing a sign language database including a sign image corresponding to each of a plurality of speech segments; a video input device for capturing an original image of a subject; an audio input device for receiving speech from the subject; a speech analyzer coupled to said audio input device, said speech analyzer identifying the database sign image corresponding to a received speech segment; a processor coupled to said memory, said speech analyzer and said video input device, said processor calculating a modification parameter, said modification parameter indicating how a database sign image should be modified to match a characteristic of an original image; a data encoder coupled to said processor, said encoder encoding a signal including an identification of a database sign image corresponding to a received speech segment, a modification parameter for the sign image, and a corresponding original image for transmission over a transmission medium to a remote location.
-
-
21. A decoding system for generating a synthesized signing image based on a received signal, said decoding system comprising:
-
a memory storing a sign language database including a sign image corresponding to each of a plurality of speech segments; a data decoder receiving a signal via a transmission medium, said signal comprising an identification of a database sign image, a modification parameter for the sign image, and a corresponding original image of a subject; an image processor coupled to the memory and the data decoder, said image processor modifying the database sign image identified in the received signal, said sign image being modified based on said received modification parameter, said image processor superimposing the modified sign image on the received original image; a display coupled to the image processor, said display displaying a synthesized signing image of the modified sign image superimposed on the original image.
-
-
22. An apparatus for generating a synthesized signing image comprising the steps of:
-
means for storing a sign language database including a sign image corresponding to each of a plurality of speech segments; means for obtaining an original image of a subject; identifying a characteristic of the subject in the original image; means for receiving a speech segment spoken by the subject; means for identifying one of said database sign images corresponding to the speech segment spoken by the subject; means for modifying the identified sign image to match the characteristic of the subject in the original image; and means for superimposing the modified sign image on the original image of the subject to generate a synthesized signing image.
-
Specification