×

Behavior recognition system and method by combining image and speech

  • US 8,487,867 B2
  • Filed: 12/09/2009
  • Issued: 07/16/2013
  • Est. Priority Date: 11/10/2009
  • Status: Expired due to Fees
First Claim
Patent Images

1. A behavior recognition system by combining an image and a speech, comprising:

  • a database, for storing a plurality of image-and-speech relation modules, wherein each of the image-and-speech relation modules comprises a feature extraction parameter and an image-and-speech relation parameter;

    a data analyzing module, for substituting a gesture image and a speech data corresponding to each other into each feature extraction parameter to obtain a plurality of image feature sequences and a plurality of speech feature sequences, and substituting each image feature sequence and each speech feature sequence corresponding to a same image-and-speech relation module into each image-and-speech relation parameter, so as to calculate a plurality of image-and-speech status parameters, wherein each image feature sequence comprises a plurality of image frame data, and the image frame data forms a plurality of image frame status combinations;

    each speech feature sequence comprises a plurality of speech frame data, and the speech frame data forms a plurality of speech frame status combinations, when the data analyzing module calculates each one of the image-and-speech status parameters, the data analyzing module substitutes each image frame status combination and each speech frame status combination into the image-and-speech relation parameter corresponding to the same image-and-speech relation module to calculate a plurality of image-and-speech sub-status parameters and selects one image-and-speech sub-status parameter from the plurality of image-and-speech sub-status parameters to serve as the image-and-speech status parameter corresponding to the image-and-speech relation module; and

    a calculating module, for using the image feature sequences, the speech feature sequences, and the image-and-speech status parameters to calculate a recognition probability corresponding to each of the image-and-speech relation modules, and taking a target parameter from the recognition probabilities.

View all claims
  • 1 Assignment
Timeline View
Assignment View
    ×
    ×