METHOD AND SYSTEM FOR EXTRACTING MESSAGES

• US 20100318360A1
• Filed: 06/10/2009
• Published: 12/16/2010
• Est. Priority Date: 06/10/2009
• Status: Active Grant

• Current Assignee: Toyota Motor Corporation
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Uehara, Yasuo
• Granted Patent: US 8,452,599 B2
• Time in Patent Office: Days
• US Class Current: 704/260
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/027 Concept to speech synthesisers; Generation of natural phrases from machine-based concepts generation of parameters for speech synthesis out of text G10L13/08
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features

Abstract

The present invention is a method and system for extracting messages from a person using the body features presented by a user. The present invention captures a set of images and extracts a first set of body features, along with a set of contexts, and a set of meanings. From the first set of body features, the set of contexts, and the set of meanings, the present invention generates a set of words corresponding to the message that the person is attempting to convey. The present invention can also use the body features of the person in addition to the voice of the person to further improve the accuracy of extracting the person'"'"'s message.

Claims

1. A message extraction system comprising:

an image capture unit obtaining a set of images;

a first features extraction unit communicatively coupled to the image capture unit to extract a first set of body features from the generated set of images;

a words extraction unit communicatively coupled to the first features extraction unit to generate a first set of words from the first set of body features;

a second features extraction unit communicatively coupled to the words extraction unit to generate a second set of body features from the generated first set of word; and

a features comparator communicatively coupled to the first features extraction unit and the second features extraction unit to determine whether the first set of body features matches the second set of body features and outputting the generated first set of words when the first set of body features matches the second set of body features.

2. The system of claim 1 further comprising a voice synthesizer unit communicatively coupled to the features comparator to generate an audio signal corresponding to the generated first set of words.

3. The system of claim 1 further comprising a meanings extraction unit communicatively coupled to the first features extraction unit to generate a meaning from the first set of body features.

4. The system of claim 3 further comprising a contexts extraction unit communicatively coupled to the meanings extraction unit to generate a context from the first set of body features.

5. The system of claim 1 wherein the first set of body features includes facial expressions.

6. The system of claim 1 wherein the first set of body features includes hand gestures.

7. The system of claim 1 further comprising:

an audio data processing system receiving an audio input and generating a second set of words based on the audio input; and

a words integrator to integrate the generated first set of words and the generated second set of words and generate an integrated set of words.

8. The system of claim 7 further comprising a voice synthesizer unit communicatively coupled to the words integrator to generate an audio signal corresponding to the integrated set of words.

9. The system of claim 7 further comprising a subtitle unit communicatively coupled to the words integrator to generate a set of subtitles corresponding to the integrated set of words.

10. The system of claim 1 further comprising a features database communicatively coupled to the first features extraction unit storing body features used to generate the first set of body features.

11. A message extraction system comprising:

a visual data processing system obtaining a set of images and generating a first set of words based on the set of images;

an audio data processing system obtaining a set of audio inputs and generating a second set of words based on the set of audio inputs; and

a words integrator communicatively coupled to the visual data processing system and the audio data processing system to integrate the generated set of first words and the generated second set of words to generate an integrated set of words.

12. The system of claim 11 further comprising a voice synthesizer unit communicatively coupled to the words integrator to generate an audio signal corresponding to the integrated set of words.

13. The system of claim 11 further comprising a subtitle unit communicatively coupled to the words integrator to generate a set of subtitles corresponding to the integrated set of words.

15. The system of claim 11 further comprising an image input unit communicatively coupled to the visual data processing system to generate the set of images.

16. The system of claim 15 wherein the generated set of image corresponds to body features.

17. A method for extracting a message comprising:

obtaining a set of images;

extracting a first set of body features corresponding to the set of images;

generating a first set of words from the first set of body features;

generating a second set of body features from the first set of words;

comparing the first set of body features to the second set of body features to determine whether the first set of body features matches the second set of body features; and

outputting the first set of words when the first set of body features matches the second set of body features.

18. The method of claim 17 further comprising the step of generating an audio signal corresponding to the first set of words.

19. The method of claim 17 further comprising the steps of:

receiving a set of audio inputs and generating a second set of words based on the set of audio input;

integrating the first set of words with the second set of words to generate an integrated set of words; and

generating an audio signal corresponding to the integrated set of words.

20. The method of claim 17 further comprising the step of generating a set of subtitles corresponding to the first set of words.